Etherboot Developers Manual Ken Yap, v5.0.7, 31 July 2002 This document explains the internals of the Etherboot package. The information here applies to version 5.0 of Etherboot. ______________________________________________________________________ Table of Contents 1. About this Developers Manual 1.1 Purpose / Scope of this Developers Manual 1.2 Obtaining the most recent version of this document 1.3 Document history 1.4 Feedback 1.5 Copyrights and Trademarks 1.6 Acknowledgements and Thanks 2. Characteristics of Etherboot 3. The execution environment of Etherboot 3.1 The network booting process 3.2 To compress or not to compress, that is the question 3.3 Real and protected mode 4. The architecture of Etherboot 4.1 Etherboot core 4.2 Drivers 4.3 Miscellaneous 4.4 External auxiliary programs 5. The development environment of Etherboot 6. Frequently asked questions 6.1 How portable is Etherboot? 6.2 How can I get Etherboot to boot a Cardbus (PCMCIA) or a USB NIC? 7. Writing an Etherboot Driver 7.1 Preliminaries 7.2 Background information 7.3 Structure of the code 7.4 Booting the code from a floppy 7.5 Booting the test code with another Etherboot ROM 7.6 Writing the code 7.7 Things to watch out for 7.8 Tidying up 8. A potted history of Etherboot ______________________________________________________________________ 11.. AAbboouutt tthhiiss DDeevveellooppeerrss MMaannuuaall 11..11.. PPuurrppoossee // SSccooppee ooff tthhiiss DDeevveellooppeerrss MMaannuuaall This document explains the internals of the Etherboot package. The information here applies to version 5.0 of Etherboot. 11..22.. OObbttaaiinniinngg tthhee mmoosstt rreecceenntt vveerrssiioonn ooff tthhiiss ddooccuummeenntt This document and related documents are also kept online at the Etherboot Home Page . This will in general have the latest source distributions and documentation. 11..33.. DDooccuummeenntt hhiissttoorryy 55..00..77 22000022--0077--3311 Released as Etherboot-doc 5.0.7. 55..00..66 22000022--0033--3311 Released as Etherboot-doc 5.0.6. 55..00..55 22000011--1122--3311 Released as Etherboot-doc 5.0.5. 55..00..44 22000011--0099--1166 Released as Etherboot-doc 5.0.4. 55..00..22 22000011--0066--2233 Released as Etherboot-doc 5.0.2. No significant changes from 5.0.1 really. 55..00..11 22000011--0055--0044 Released as Etherboot-doc 5.0.1. 55..00..00 22000011--0044--2266 Released with Etherboot 5.0.0. 11..44.. FFeeeeddbbaacckk Comments on and corrections for this Developers Manual may be directed to the primary author . 11..55.. CCooppyyrriigghhttss aanndd TTrraaddeemmaarrkkss (C) 2001,2002 Ken Yap. This manual may be reproduced in whole or in part, without fee, subject to the following restrictions: +o The copyright notice above and this permission notice must be preserved complete on all complete or partial copies. +o Any translation or derived work must be approved by the author in writing before distribution. +o If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided. +o Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given. Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. All trademarks mentioned in this document belong to their respective owners. 11..66.. AAcckknnoowwlleeddggeemmeennttss aanndd TThhaannkkss Thanks to all the people who have contributed information and corrections to this document. 22.. CChhaarraacctteerriissttiiccss ooff EEtthheerrbboooott For an introduction to Etherboot from the user perspective you may wish to read the User Manual first. In order to understand Etherboot development, it is necessary to first understand how Etherboot differs from normal application programs. 1. Etherboot runs as a standalone program, not under an operating system. The only services it relies on are those provided by the BIOS of the motherboard. 2. It has complete access to the "bare metal" of the machine. 3. It needs to have as small a memory footprint as practicable, in order to maximise the amount of memory available to the code that it loads. 33.. TThhee eexxeeccuuttiioonn eennvviirroonnmmeenntt ooff EEtthheerrbboooott 33..11.. TThhee nneettwwoorrkk bboooottiinngg pprroocceessss Since this is the part that the user sees first, let us first demystify how network booting works. From time immemorial, well actually since the IBM XT appeared on the market, the PC architecture has a mechanism for invoking "extension BIOSes". The original reason for this mechanism was to allow adaptor cards that the main BIOS didn't know how to deal with to carry ROMs with initialisation code or drivers. An early example was the XT hard disk controller. The main BIOS of XTs only knew how to boot from floppies. When an XT hard disk controller is added, the code in the ROM on the controller appears in the memory space of the PC and is called as part of the machine initialisation. Another example is the BIOSes on VGA video adaptor cards, although strictly speaking that is a special case in terms of ROM address. When network adaptors were made for the PC, it was a natural step to put ROMs on them that could contact a server for network booting. How does the main BIOS know that the code in the ROM is to be executed and why does it not execute some random code by accident? The ROM code has several conditions placed on it. +o The ROM must start on a 2kB boundary in the memory space, between 0xC8000 and 0xEE000, although some main BIOSes scan outside these limits. +o The first two bytes of the ROM must be 55 AA hex. +o The third byte of the ROM should contain the number of bytes in the ROM code divided by 512. So if the ROM code is 16kB long, then this byte would hold 20 hex (32 decimal). +o All the bytes in the ROM (specified by the length byte just mentioned) must checksum to 8 bits of binary zero. The sum is formed by 8 bit addition of all the bytes, throwing away the carry. Note that there is not a particular location designated as the "checksum byte". Normally the ROM building process alters an unused byte somewhere to fulfil the checksum condition. If such a ROM is detected and validated by a scan, then the main BIOS does a far call to ROMSEG:3, where ROMSEG is the segment of the ROM and 3 is the offset to transfer control to the discovered extension BIOS. Typically a network boot ROM does not take full control at this point. Instead the normal procedure to do some initialisation or probing of the hardware and then plant a vector that will be called when the BIOS is ready to boot the OS. The vector used for this purpose is normally interrupt 0x19 although interrupt 0x18 is sometimes used. For PCI plug and play ROMs things are more complicated. For the full story, you need to get the specifications from Phoenix and Intel. Here is a quick summary. +o The boot ROM must satisfy the requirements for ROMs listed above (called legacy ROMs). +o There are two additional structures in the ROM, the PCIR structure and the PnP structure. These structures are pointed to by offsets in two 16-bit words at 0x18 and 0x1A bytes respectively from the beginning of the ROM. As a double check, the structures each begin with 4 magic bytes, PCIR and $PnP respectively. +o The PCIR structure contains the vendor and device IDs of the network adaptor, and these must match the IDs that is stored in the adaptor's PCI configuration memory, or the ROM will be ignored. +o The PnP structure contains various vectors. The one of interest to us is the Boot Execution Vector (BEV). This points to the starting point of the boot ROM code. The first time the ROM is detected, it is called at the ROMSEG+3 entry point as for legacy ROMs. This entry point must indicate, by returning 0x20 in register AX, that it is a network boot device. When the BIOS is ready to boot, it calls the BEV. Note that the BIOS only calls the BEV if the BIOS configuration specifies the device in the boot sequence. +o There is a checksum for the PnP structure in addition to the overall checksum in legacy ROMs. The network boot process then works like this: 1. The main BIOS detects the Etherboot ROM as an extension BIOS and passes control to it with a far call. 2. For legacy ROMs, the Etherboot code hooks itself to interrupt 0x19 and returns control to the main BIOS. For PnP ROMs the Etherboot code indicates that it is a bootable device. 3. The main BIOS finishes initialising other devices and boots the operating system by calling interrupt 0x19. 4. The Etherboot code gains control. 5. It initialises the network hardware so that it is ready to send and receive packets. 6. It sends a Boot Protocol (BOOTP ) or Dynamic Host Configuration Protocol (DHCP ) broadcast query packet. An alternative is Reverse Address Resolution Protocol (RARP ) 7. Assuming a reply is received, the Etherboot code decodes the fields of the reply, sets its IP address and other parameters, and sends a Trivial File Transfer Protocol (TFTP ) request to download the file. Be aware that the 16-bit counter field of TFTP may limit transfers to 16 MB (signed interpretation), 32 MB (unsigned interpretation) or 90 MB (unsigned, large block size option active). This is a rollover bug in many TFTP servers, but quite prevalent. An alternative loading protocol is Network File System (NFS ) protocol. In this instance a mount of the remote filesystem is done (with the bare minimum of features) and the boot file is read off the filesystem. 8. The file to be loaded is in a special format, it contains a "directory" in the first block that specifies where in memory the various pieces of the file are to be loaded. Formats that are supported are tagged or Execution and Loader Format (ELF) . One small extension to ELF has been made, the top bit in the e_flags longword of the ELF header has been used as ELF_PROGRAM_RETURNS_BIT, meaning that the program transferred to intends to return to Etherboot, e.g. a menu program, and that Etherboot should not disable the network interface yet. See the file osloader.c. Eric Biederman adds: The ELF format is actually docmented in the SysV Generic ABI doc. http://www.sco.com/developer/devspecs/gabi41.pdf http://www.sco.com/developer/devspecs/abi386-4.pdf Also check out the linux standard base it has good links to all of these documents, in its related documents section. http://www.linuxbase.org/spec/ 9. Etherboot transfers control to the loaded image. Notice no assumption was made that the image is a Linux kernel. Even though loading Linux kernels is the most common use of Etherboot, there is nothing in the procedure above that is Linux specific. By creating the loaded file appropriately, different operating systems, e.g. FreeBSD, DOS, can be loaded. In the case of a Linux kernel, there is some additional work to be done before the kernel can be called, so the segment of the file that Etherboot transfers to is not the startup segment of the kernel, but an initial stub, whose code is in mknbi/first32.c. This stub has several tasks, which either cannot be done by Etherboot, or should not be done by Etherboot because they are Linux specific. These are: to append kernel arguments from option 129 of the BOOTP or DHCP reply; to copy and expand special kernel parameters, in particular the vga= and the ip= parameters and then to point the kernel to the location of the parameter area; to move the RAMdisk, if there is one, to the top of memory (this last cannot be done at image creation time since the size of the RAM of the machine is not known then). The kernel parameters are passed to the kernel as a pointer to the string written in a certain location in the original bootblock (boot.S from the Linux kernel sources). This is a 16-bit pointer and is the offset of the parameter area from the base of the bootblock. This is one reason why the parameter area must be in the same 64kB as the bootblock. If the components of Etherboot are to be relocated elsewhere, e.g. 0x80000 upwards, then they should be relocated together. In version 0x0202 and above of the Linux setup segment, this can be passed instead as a absolute 32-bit pointer in a certain location in the setup segment. This eases the relocation requirements. The address of the RAM disk, if it exists, is passed to the kernel as a 32-bit address in a certain location in the setup segment (setup.S from the Linux kernel sources). This is filled in with the final location of the RAM disk after it has been moved to the top of memory. 33..22.. TToo ccoommpprreessss oorr nnoott ttoo ccoommpprreessss,, tthhaatt iiss tthhee qquueessttiioonn We simplified things a little when we talked about how the main BIOS detects the Etherboot ROM and passes control to it. At this point the code is executing from ROM. There are two problems with executing from ROM where x86 PCs are concerned. 1. The x86 architecture does not easily support Position Independent Code (PIC). The main drawback when executing C code is referencing global entities (global and static variables). Since the ROM address is not known when building the image, addresses cannot be assigned to global entities. More advanced environments have a dynamic loader for adjusting references just before use. Etherboot has no such help. If the code were written in assembler, we could use a convention like always referencing global entities as offsets from a particular register. But we don't want to write in assembler and we don't have control over what the C compiler generates. 2. C code assumes that data locations are writable. ROM locations are not writable, by definition. One could remedy this by locating the writable entities in a RAM segment, but this causes more complication. For the reasons above, Etherboot copies itself into a known RAM area and executes from there. The area chosen is the 64kB segment starting at 0x90000. The area from 0x90000 to 0x93FFF is reserved for various code and data structures needed by the kernel and the Etherboot code starts at 0x94000. This gives us about 48kB of room for code, data, initialised data and stack. There is no heap; Etherboot does not have dynamic storage allocation. This keeps things simple, makes it a bit less prone to programming errors, and also acts as a check against unthinking use of memory by programmers. One thing that we usually want to do is minimise the size of the ROM used to hold Etherboot. Even if the network adapter accommodates large ROMs, there are many claims on the area between 0xC8000 to 0xEE000, by other extension BIOSes, by peripherals that use shared memory, and so forth. Etherboot allows the code to be compressed before loading into ROM, and then when the ROM is executed, a special header decompresses the code into memory. ROM images that are compressed are designated by a suffix of .lzrom. By the magic or horror, depending on how you view it, of conditional code using, or abusing, the C preprocessor, both the normal and the decompressing loaders are generated from one source file, loader.S. The compressor is a C program called lzhuf.c. So to summarise, if the loader is normal, it simply copies the payload, i.e. the bytes after itself, to the final execution environment and jumps to it. If it is a decompressing loader, it copies all of the ROM contents to a temporary location starting at 0x80000 and continues execution from there. The reason for this first relocation is so that execution goes faster. Typically ROM has longer access times than RAM and is often accessed 8 bits at a time. The glue chipsets used in motherboards add wait states so that the CPU can interface with relatively slow ROM. This is to reduce ROM costs, as execution speed is not important at boot time. By copying itself to RAM first, the decompression goes faster. After relocation and resumption of execution, the loader decompresses the payload into the final execution location and jumps to it. If you have understood by now that the loader must be written as PIC and in assembler, give youself a pat on the back. 33..33.. RReeaall aanndd pprrootteecctteedd mmooddee One of the complications of the x86 architecture is the existence of the real and protected modes of execution. To simplify a long story, if we want to execute 32-bit code as generated by the C compiler, we need to be in protected mode. However the processor boots up in real mode, which is 16-bit. So the loaders must execute in real mode. In addition, the main BIOS calls the extension BIOS in real mode. So at least the prologue of the main code must be in real mode. BIOS calls must also be in real mode. In Etherboot this is handled by a pair of functions (written in assembler of course) called prot_to_real and real_to_prot that do the switching. The C code doesn't call this directly. They are implicitly called from routines that use BIOS services, such as printing characters to the screen or reading characters from the keyboard. In previous versions of Etherboot, the 16-bit code was written for as86/nasm. Fortunately they have now been translated to assemble using the .code16 mode of the GNU assembler, so less extra tools are needed now. In the case of Linux tagged images, the initial segment first32.c runs in protected mode. The segment uses the same stack as Etherboot. Since the call to the initial segment goes through real mode first, this means that the stack must be located in the same 64kB segment as the Etherboot body. To be specific, say the Etherboot code runs with a base of 0x94000. The stack is near the end of the low 640kB of memory, i.e. 0xA0000. This means that the real mode stack segment register (%ss) has the value 0x9400 and the real mode stack pointer (%sp) is a little less than 0xC000 (the difference between the top of memory and the base of the Etherboot segment). When first32.c is called, the code at the beginning does a calculation to work out what the 32-bit extended stack pointer (%esp) should be set to. This is done by adding the 16-bit stack pointer (%sp) to the relocation base of Etherboot (%ss * 16). This calculation will not produce the correct result if first32.c is not in the same 64kB as Etherboot. Currently first32.c is loaded at 0x92800 so this condition is satisfied. But it will fail if first32.c is called from something other than Etherboot, e.g. from another boot ROM, or if mknbi.pl has been changed to load first32.c somewhere else, e.g. 0x82800. The remedy for this situation is for first32.c to establish its own stack, and to copy the arguments on the stack that are passed from Etherboot from the old stack. The program menu-simple.c which is called by the initial code in startmenu.S uses this strategy. Not only do you have to copy the arguments, but you also have to establish another GDT, because the old GDT will be using a different 16-bit segment. Recall that the mode switching routines prot_to_real and real_to_prot assume that protected mode and real mode share the same stack. 44.. TThhee aarrcchhiitteeccttuurree ooff EEtthheerrbboooott Leaving aside the peripheral code for the loaders, the rest of Etherboot can be divided into the core, the drivers and miscellaneous (catch-all category). ______________________________________________________________________ +------------------+------------------+ | Etherboot core | Miscellaneous | +------------------+------------------+ | Driver support | +-------------------------------------+ | Hardware drivers | +-------------------------------------+ | Hardware | +-------------------------------------+ ______________________________________________________________________ 44..11.. EEtthheerrbboooott ccoorree The core of Etherboot handles the protocol for obtaining a network identity and for loading data over the network. This comprises the files start32.S (startup code and BIOS interface), main.c (most of the protocol handling), config.c (links in the desired driver), osloader.c (support for various load image formats), nfs.c (support for NFS I/O) and misc.c (printing routines and A20 gate handling). config.c is special in that it is compiled into one object file for each network adapter, again using C preprocessor technology. 44..22.. DDrriivveerrss These files are listed in the file NIC. Generally each file represents a family of network adapters. For example tulip.c handles all adapters that use a Tulip compatible network controller, even if they are from different manufacturers. 3c90x.c handles a whole family of related adapters from 3Com. The interface between the core and the drivers is well-defined and explained in the section on ``writing a driver''. The files timer.h and timer.c provide routines for access to the second hardware timer of the PC. This is used for implementing microsecond timeouts. The files pci.h and pci.c provide a PCI initialisation subsystem that is executed for PCI adapters. skel.c is a skeleton driver that an aspiring driver writer can use as a starting point. 44..33.. MMiisscceellllaanneeoouuss In this category are all the files that don't fit into the first two categories. These are ansisec.c (fancy graphics), bootmenu.c (menu support), floppy.c (floppy I/O), md5.c (authentication), and serial.S (serial console support). 44..44.. EExxtteerrnnaall aauuxxiilliiaarryy pprrooggrraammss In Etherboot 5.0 and later there is the facility to load and run external auxiliary programs. Etherboot executes at 0x94000. All the memory below that and above 0x10000 is fair game for loading. What if we did not load the target operating system but instead loaded an external program, one that returned to Etherboot after doing something. This something could be a fancy menu system. The advantage of this approach is that the menu system does not have to be compiled into Etherboot, which means it can be changed without changing ROMs, and can be much larger. How does such a menu program indicate to Etherboot which image is to be loaded next? Several new features of Etherboot make this possible. Firstly, a bit in the header of the loaded image is used to indicate that the loaded program intends to return to Etherboot, as opposed to never returning, in the case of an operating system. Next, the external program is passed a pointer to the BOOTP/DHCP structure obtained by Etherboot, which includes the filename field. Finally, the external program can return one longword of status to Etherboot to indicate various conditions. So here's how the external menu program would work. Etherboot is told to load an image which is a menu program. The program would present the user with a list of images to choose from. The presentation can be as simple as a numbered menu or as fancy as a point and click interface (if you can arrange to interface to the pointer). When a particular image has been chosen, the menu program alters the filename field in the BOOTP/DHCP structure and returns a longword of status indicating that Etherboot should retry the loading. Since the filename has been altered, Etherboot will end up loading the desired image instead of the menu this time around. Think of it as a program chaining facility. 55.. TThhee ddeevveellooppmmeenntt eennvviirroonnmmeenntt ooff EEtthheerrbboooott Etherboot is written in C, with certain routines written in x86 assembler for access to machine resources or to do things not possible in C. The tools used are the GNU C compiler and GNU x86 assembler. ld is used to link the object files. objcopy is used to convert the output of the linker to pure binary, suitable for standalone execution. An auxiliary program called makerom converts this binary into a form suitable for writing into a ROM. A Makefile coordinates all the building procedures. However there are many network adapters that differ only in the PCI vendor and device IDs. This information must be programmed into the ROM header using makerom. An auxiliary script called genrules.pl takes a file NIC, containing a tabular description of all the device names and IDs and outputs make rules to a file called Roms that is included by the main Makefile to generate all the ROM images. Another file included by Makefile is Config, which contains user configurable build parameters. Let us take an example of a ROM image that we could build, mx987x5.lzrom. According to this line in the file NIC: ______________________________________________________________________ mx987x5 tulip 0x10d9,0x0531 ______________________________________________________________________ the driver code is in tulip.c and the vendor ID is 0x10d9 and the device ID is 0x0531. If one were to say make bin32/mx987x5.lzrom, the following actions happen: +o Any needed utilities such as bin/makerom and bin/lzhuf are built. +o tulip.c is compiled to bin32/tulip.o. +o The startup routine, start32.S is assembled to bin32/start32.o. +o All the core Etherboot files, e.g. main.c, osloader.c, etc, are compiled to corresponding object files in bin32/*.o and combined into an ar archive, bootlib.a, for convenience. +o The file config.c is compiled to bin32/config-tulip.o while applying the appropriate define: INCLUDE_TULIP. +o The startup routine (which must be first in the list of objects), the driver object, bootlib.a, and config-tulip.o are linked to produce an ELF binary, linked to run at the relocation address of Etherboot, currently 0x94000. +o The ELF binary is converted to a pure binary using the GNU utility objcopy. +o The prepended loaders, rloader, rzloader, prloader, and przloader, are produced by assembling loader.S with different defines. The meaning of the prefixes are: rloader = basic ROM loader, rzloader = decompressing ROM loader, prloader = PCI header basic ROM loader, przloader = PCI header decompressing ROM loader. The loaders contain the needed headers for the BIOS to recognise the code as an extension ROM. See the ``discussion'' on the extension BIOS mechanism. +o In the case of mx987x5.lzrom, this is for a PCI adaptor and we want a compressed ROM, so the loader is przloader. This is prepended to the binary image to generate a ROM image. +o makerom is run over this ROM image to pad it to the size of a standard EPROM. (8 kB, 16 kB, 32 kB, etc.) The PCI IDs are written into the image into the PnP structure at the right spots. A string identifying the device and Etherboot version is written into the unprogrammed bytes at the end of the ROM image, if space is available, and a pointer to that written into the PnP structure. The vendor string and the device string are normally printed by the PCI PnP BIOS at boot time. Makerom also calculates the checksum (both for the PnP structure, if present and for the whole image) and alters designated spare bytes in the image so that the checksums come out right. +o The image is ready to be written into an EPROM. However if the image is to be tested from floppy, hard disk or LILO/SYSLINUX, additional preloaders are prepended to the ROM image. In the case of floppy or hard disk loading, the preloader is called bin/boot1a.bin. It is one sector (512 bytes) long and loaded from the floppy or hard disk, starting at the beginning of the floppy or the beginning of a hard disk partition. It then loads the blocks following itself, which it assumes to be an Etherboot image, into memory, then jumps to it at a second entry point 6 bytes from the beginning of the image. This second entry point is used when not booting from ROM. In the case of LILO/SYSLINUX booting, the preloader is liloprefix.bin. It contains what looks like a floppy sector and a startup segment to LILO/SYSLINUX and makes LILO/SYSLINUX "think" that this is a Linux kernel. +o There is yet another preloader and this is not for a device but for an OS environment. This is comprefix.bin, and when prepended to a ROM image, it makes it look like a DOS .com executable, including the peculiarity of starting at COM_SEGMENT:0x100. All it does however is jump to the Etherboot ROM image, again at offset 6. +o All the preceeding special preloaders have associated Makefile rules and are created by asking for images with the appropriate suffix. For example, if one wanted an image for writing onto a floppy one would say: ___________________________________________________________________ make mx987x5.lzdsk ___________________________________________________________________ 66.. FFrreeqquueennttllyy aasskkeedd qquueessttiioonnss 66..11.. HHooww ppoorrttaabbllee iiss EEtthheerrbboooott?? Now and then, the question is raised whether Etherboot is portable to other machine platforms, i.e. not x86. Etherboot is not per se a portable program as it only handles one architecture. Portability was not a goal simply because the only development platform available to me that did not already have a boot ROM capability was the x86 platform. (Sparcs and Alphas already have network booting capability in their boot ROMs.) The portability, or rather, lack of portability, of Etherboot isn't simply an issue of the assembler code. Etherboot needs and assumes certain hooks into the running environment, e.g. the extension BIOS mechanism that invokes Etherboot, the BIOS calls made by Etherboot, and the memory layout of the Etherboot components. The core routines of Etherboot that handle the address configuration and network loading should be machine-independent, but there may be byte-order bugs lurking, this has not been checked. The drivers are hardware dependent of course, so the ISA adaptor drivers are no use anywhere else. The PCI adaptor drivers should work on other machines, but the PCI subsystem that handles adaptor detection is x86 PCI BIOS specific. Here are the low-level services that need to provided: writing and reading from the console, determining the size of memory, obtaining the time of day, inserting a boot vector to be called, and a microsecond timer for short delays. Depending on the target environment, these services may be provided in different ways. 66..22.. HHooww ccaann II ggeett EEtthheerrbboooott ttoo bboooott aa CCaarrddbbuuss ((PPCCMMCCIIAA)) oorr aa UUSSBB NNIICC?? Cardbus (PCMCIA) and USB NICs are interfaced to the CPU through their respective bus controllers. Before Etherboot can address the registers and memory on these NICs, it must initialise the bus controller appropriately. In Linux this is done by the PCMCIA and USB subsystems. In Etherboot, all this must be done by the code in the ROM. What needs to be written is a subsystem like the PCI subsystem in Etherboot. Volunteers are most welcome. 77.. WWrriittiinngg aann EEtthheerrbboooott DDrriivveerr 77..11.. PPrreelliimmiinnaarriieess So Etherboot does not have a driver for your network adapter and you want to write one. You should have a good grasp of C, especially with respect to bit operations. You should also understand hardware interfacing concepts, such as the fact that the x86 architecture has a separate I/O space and that peripherals are commanded with `out' instructions and their status read with `in' instructions. A microprocessor course such as those taught in engineering or computer science curricula would have given you the fundamentals. (Note to educators and students in computer engineering: An Etherboot driver should be feasible as a term project for a final year undergraduate student. I estimate about 40 hours of work is required. I am willing to be a source of technical advice.) Next you need a development machine. This can be your normal Linux machine. You need another test machine, networked to the development machine. This should be a machine you will not feel upset rebooting very often. So the reset button should be in working condition. :-) It should have a floppy drive on it but does not need a hard disk, and in fact a hard disk will slow down rebooting. Alternatively, it should have another network adapter which can netboot; see discussion further down. Needless to say, you need a unit of the adapter you are trying to write a driver for. You should gather all the documentation you can find for the hardware, from the manufacturer and other sources. 77..22.. BBaacckkggrroouunndd iinnffoorrmmaattiioonn There are several types of network adapter architecture. The simplest to understand is probably programmed I/O. This where the controller reads incoming packets into memory that resides on the adapter and the driver uses `in' instructions to extract the packet data, word by word, or sometimes byte by byte. Similarly, packets are readied for transmission by writing the data into the adapter's memory using `out' instructions. This architecture is used on the NE2000 and 3C509. The disadvantage of this architecture is the load on the CPU imposed by the I/O. However this is of no import to Etherboot (who cares how loaded the CPU is during booting), but will be to Linux. Next in the sophistication scale are shared memory adapters such as the Western Digital or SMC series, of which the WD8013 is a good example. Here the adapter's memory is also accessible in the memory space of the main CPU. Transferring data between the driver and the adapter is done with memory copy instructions. Load on the CPU is light. Adapters in this category are some of the best performers for the ISA bus. Finally there are bus mastering cards such as the Lance series for the ISA bus and practically all good PCI adapters (but not the NE2000 PCI). Here the data is transferred between the main memory and the adapter controller using Direct Memory Access. Setting up the transfers usually involves a sequence of operations with the registers of the controller. 77..33.. SSttrruuccttuurree ooff tthhee ccooddee Examine the file skel.c, in the src directory, which is a template for a driver. You may also want to examine a working driver. You will see that an Etherboot driver requires 5 functions to be provided: +o A probe routine, that determines if the network adapter is present on the machine. This is passed a pointer to a `nic' struct, a list of candidate addresses to probe, and possibly a pointer to a `pci' struct. This routine should initialise the network adapter if present. If a network adapter of the type the driver can handle is found, it should save the I/O address at which it was found for use by the other routines. In the case of ISA adapters, it may be passed a list of addresses to try, or if no list is passed in, it may use an internal list of candidate addresses. In the case of PCI adapters, the address has already been found by the PCI support routines. Then it should determine the Ethernet (MAC) address of the adapter and save it in nic->node_addr. It should then call the reset routine to initialise the adapter. Finally it should fill in the function pointers for the other routines, and return the `nic' pointer. If it fails to find an adapter, it should return 0. The field rom_info in the `nic' struct contains, on entry to the probe routine, a pointer to the structure rom_info, defined in ehterboot.h. This contains the segment address and length in shorts of the ROM as detected by the BIOS. These can be used by the driver to discriminate between multiple instances of the same network adaptor on the bus, and choose to activate only the one the ROM is installed on. For example, the 3c509 hardware has a register indicating what address the ROM is mapped at. Note that the driver should not reject "ROM segment addresses" outside 0xC000 to 0xEE00 as this indicates booting from floppy disk or other non-ROM media. Initialising the adapter means programming the registers so that the chip is ready to send and receive packets. This includes enabling the appropriate hardware interface (10B2, 10BT) in the case of adapters with more than one interface, and setting the right speed (10Mb, 100Mb) if the hardware does not autosense and set it. It also includes setting up any memory buffers needed by the hardware, along with any necessary pointers. Note that you should program the receiver registers to allow broadcast Ethernet packets to be received. This is needed because other IP hosts will do an ARP request on the diskless computer when it boots. +o A reset routine, that resets the adapter to an initial state. This is passed a pointer to a `nic' struct. This can be called from the probe routine. +o A disable routine, which puts the adapter into a disabled state. This is passed a pointer to a `nic' struct. This is needed to leave the adapter in a suitable state for use by the operating system which will be run after Etherboot. Some adapters, if left in an active state, may crash the operating system at boot time, or cannot be found by the operating system. +o A transmit routine, to send an Ethernet packet. This is passed a pointer to a `nic' struct, the 6 byte Ethernet address of the destination, a packet type (IP, ARP, etc), the size of the data payload in bytes, and a pointer to the data payload. Remember the packet type and length fields are in x86 byte order (little-endian) and the adapter's byte order may be the reverse (big-endian). Note that the routine knows nothing about IP (or any other type) packets, the data payload is assumed to be a filled in packet, ready to transmit. +o A poll routine, to check if a packet has been received and ready for processing. This is passed a pointer to a `nic' struct. If a packet is available, it should copy the packet from the adapter into the data area pointed to by nic->packet, and set nic->packetlen to the length of the data, and return 1, otherwise 0. A few Ethernet controller chips will receive packets from itself, as detected by having a source address of itself. You can throw these out immediately on reception and not bother the upper layer with them. Only the probe routine needs to be public, all the other routines should be static and private to the driver module. Similarly all global data in the driver should be static and private. In cards.h add a prototype for the probe routine, conditional on INCLUDE_NAMEOFNIC. NAMEOFNIC is derived from the name of the driver source file for your adapter by uppercasing alphabets and converting hyphen to underscore. If the file is pop-sicle.c, then the symbol is INCLUDE_POP_SICLE. In config.c, add a struct entry containing the name of the driver and a pointer to the probe routine, also conditional on INCLUDE_NAMEOFNIC. The third element of the structure should be pci_probeaddrs in the case of PCI adapters, otherwise 0. If the NIC is a PCI adapter, near the top of config.c, add INCLUDE_NAMEOFNIC to the list of defines that cause INCLUDE_PCI to be set. Also add an entry in the pci_nic_list array with the name, vendor and id fields filled in. You can obtain the vendor and device ids from the file /usr/include/linux/pci.h. It is also displayed by PCI BIOSes on bootup, or you can use the lspci program from the pciutils package to discover the ids. The other fields will be filled in by the pci routines. Then add an entry to the file NIC so that the build process will create Makefile rules for it in the file Roms. The build process will cause the driver object will be pulled in from the driver library. 77..44.. BBoooottiinngg tthhee ccooddee ffrroomm aa ffllooppppyy Use the rule for bin32/driver.fd0 or the command `cat bin/boot1a.bin bin32/driver.rom > /dev/fd0' to write another instance of the driver to the floppy for testing. Use lots of printf statements to track where execution has reached and to display the status of various variables and registers in the code. You should expect to do this dance with the development machine, floppy disk and target machine many many times. 77..55.. BBoooottiinngg tthhee tteesstt ccooddee wwiitthh aannootthheerr EEtthheerrbboooott RROOMM There is another method of testing ROM images that does not involve walking a floppy disk between the machines and is much nicer. Set up a supported NIC with a boot ROM. Put the target NIC on the same machine but at a non-conflicting I/O location. That is to say, your test machine has two NICs and two connections to the LAN. Then when you are ready to test a boot image, use the utility mknbi-rom to create a network bootable image from the ROM image, and set up bootpd/DHCPD and tftpd to send this over the when the machine netboots. Using Etherboot to boot another version of itself is rather mind-boggling I know. 77..66.. WWrriittiinngg tthhee ccooddee First set up the various required services, i.e. BOOTP/DHCP, tftp, etc. on the development machine. You should go through the setup process with a supported adapter card on a test machine so that you know that the network services are working and what to expect to see on the test machine. If you are starting from a Linux driver, usually the hardest part is filtering out all the things you do not need from the Linux driver. Here is a non-exhaustive list: You do not use interrupts. You do not need more than one transmit buffer. You do not need to use the most efficient method of data transfer. You do not need to implement multicasting. You do not need to implement statistics counting. Generally speaking, the probe routine is relatively easy to translate from the Linux driver. The exception is when you need to handle media and speed switching. The reset routine is tricky because you won't know if it worked until you try to transmit or receive. So check carefully for typographical errors. The transmit is usually straightforward, and the receive a bit more difficult. The main problem is that in the Linux driver, the work is split between routines called from the kernel and routines triggered by hardware interrupts. As mentioned before, Etherboot does not use interrupts so you have to bring the work of transmitting and receiving back into the main routines. The disable routine is straightforward if you have the hardware commands. When coding, first get the probe routine working. You will need to refer to the programmer's guide to the adapter when you do this. You can also get some information by reading a Linux or FreeBSD driver. You may also need to get the reset routine working at this time. Next, get the transmit routine working. To check that packets are going out on the wire, you can use tcpdump or ethereal on the development machine to snoop on the Ethernet. The first packet to be sent out by Etherboot will be a broadcast query packet, on UDP port 67. Note that you do not need interrupts at all. You should ensure the packet is fully transmitted before returning from this routine. You may also wish to implement a timeout to make sure the driver doesn't get stuck inside transmit if it fails to complete. A couple of timer routines are available for implementing the timeout, see timer.h. You use them like this (in pseudo-code): ______________________________________________________________________ for (load_timer2(TIMEOUT_VALUE); transmitter_busy && (status = timer2_running()); ) ; if (status == 0) transmitter_timed_out; ______________________________________________________________________ The timeout value should be 1193 per millisecond of wait. The maximum value is 65535, which is about 54 milliseconds of timeout. If you just need to delay a short time without needing to do other checks during the timeout, you can call waiton_timer2(TIMEOUT_VALUE) which will load, then poll the timer, and return control on timeout. Please do not use counting loops to implement delays. Such loops are CPU speed dependent and can fail to give the right delay period when run on a faster machine. Next, get the receive routine working. If you already have the transmit routine working correctly you should be getting a reply from the BOOTP/DHCP server. Again, you do not need interrupts, unlike drivers from Linux and other operating systems. This means you just have to read the right register on the adapter to see if a packet has arrived. Note that you should NOT loop in the receive routine until a packet is received. Etherboot needs to do other things so if you loop in the poll routine, it will not get a chance to do those tasks. Just return 0 if there is no packet awaiting and the main line will call the poll routine again later. Finally, get the disable routine working. This may simply be a matter of turning off something in the adapter. 77..77.. TThhiinnggss ttoo wwaattcchh oouutt ffoorr Things that may complicate your coding are constraints imposed by the hardware. Some adapters require buffers to be on word or doubleword boundaries. See rtl8139.c for an example of this. Some adapters need a wait after certain operations. 77..88.. TTiiddyyiinngg uupp When you get something more or less working, release early. People on the mailing lists can help you find problems or improve the code. Besides you don't want to get run over by a bus and then the world never gets to see your masterpiece, do you? :-) Your opus should be released under GPL, BSD or a similar Open Source license, otherwise people will have problems using your code, as most of the rest of Etherboot is GPLed. 88.. AA ppootttteedd hhiissttoorryy ooff EEtthheerrbboooott In Linux circles Netboot appeared first. According to the docs Jamie Honan was the person who coded up the first version and specified the format of the tagged image files. This version used assembler code taken from packet drivers to interface to the hardware, only Western Digital (now SMC) NICs in the first instance. It also required a DOS environment to compile. Later on Gero Kuhlmann took over the development of Netboot and made tremendous improvements to it. Among other things he created a harness that would simulate just enough of a DOS environment so that unmodified packet driver binaries could be used in a boot ROM. This allows any NIC on the market that has a packet driver to be used immediately. He also migrated the development to a Linux (Unix) platform. Etherboot was ported from FreeBSD by Markus Gutschke. He made it compile under Linux and added code to support tagged images in addition to NFS boot. Since tagged images are a more general mechanism and requires less boot rom code, this has become the preferred loading method. Markus has also coded most of the additional features between 2.0 and 3.0, such as additional bootp tags, ANSI screen escapes, etc. Many of the features common to Etherboot and Netboot, such as the tagged image format, the support programs such as mknbi, and support in the Linux kernel for diskless booting, are by Gero or Markus. Ken Yap came to Etherboot a bit later. His original objective was to produce a 16 bit version that could be used to netboot ELKS and other OSes on older CPUs. As these things happen, he got enticed into improving the code, doing structural rearrangement, and merging contributions from others, and is now the primary maintainer of Etherboot. In the early days, the Etherboot web page was hosted by the Sydney Linux Users Group web site. Quite coincidentally and unrelatedly, Jamie Honan is one of the founding members of SLUG, so the story has come a full circle here. Since April 2000, Etherboot has been hosted at Sourceforge . Sourceforge has provided superb facilities for hosting community Open Source development.