2.3. Toolchain Technical Notes

This section explains some of the rationale and technical details behind the overall build method. It is not essential to immediately understand everything in this section. Most of this information will be clearer after performing an actual build. This section can be referred back to at any time during the process.

The overall goal of Chapter 5 is to provide a temporary environment that can be chrooted into and from which can be produced a clean, trouble-free build of the target HLFS system in Chapter 6. Along the way, we separate from the host system as much as possible, and in doing so, build a self-contained and self-hosted toolchain. It should be noted that the build process has been designed to minimize the risks for new readers and provide maximum educational value at the same time. In other words, more advanced techniques could be used to build the system.

Some key technical points of how the Chapter 5 build method works:

Binutils is installed first because the ./configure runs of both GCC and Glibc perform various feature tests on the assembler and linker to determine which software features to enable or disable. This is more important than one might first realize. An incorrectly configured GCC or Glibc can result in a subtly broken toolchain, where the impact of such breakage might not show up until near the end of the build of an entire distribution. A test suite failure will usually alert this error before too much additional work is performed.

Binutils installs its assembler and linker in two locations, /tools/bin and /tools/$TARGET_TRIPLET/bin. The tools in one location are hard linked to the other. An important facet of the linker is its library search order. Detailed information can be obtained from ld by passing it the --verbose flag. For example, an ld --verbose | grep SEARCH will illustrate the current search paths and their order. It shows which files are linked by ld by compiling a dummy program and passing the --verbose switch to the linker. For example, gcc dummy.c -Wl,--verbose 2>&1 | grep succeeded will show all the files successfully opened during the linking.

The next package installed is GCC. An example of what can be seen during its run of ./configure is:

checking what assembler to use...
        /tools/i686-pc-linux-gnu/bin/as
checking what linker to use... /tools/i686-pc-linux-gnu/bin/ld

This is important for the reasons mentioned above. It also demonstrates that GCC's configure script does not search the PATH directories to find which tools to use. However, during the actual operation of gcc itself, the same search paths are not necessarily used. To find out which standard linker gcc will use, run: gcc -print-prog-name=ld.

Detailed information can be obtained from gcc by passing it the -v command line option while compiling a dummy program. For example, gcc -v dummy.c will show detailed information about the preprocessor, compilation, and assembly stages, including gcc's included search paths and their order.

The next package installed is Glibc. The most important considerations for building Glibc are the compiler, binary tools, and kernel headers. The compiler is generally not an issue since Glibc will always use the gcc found in a PATH directory. The binary tools and kernel headers can be a bit more complicated. Therefore, take no risks and use the available configure switches to enforce the correct selections. After the run of ./configure, check the contents of the config.make file in the glibc-build directory for all important details. Note the use of CC="gcc -B/tools/bin/" to control which binary tools are used and the use of the -nostdinc and -isystem flags to control the compiler's include search path. These items highlight an important aspect of the Glibc package—it is very self-sufficient in terms of its build machinery and generally does not rely on toolchain defaults.

After the Glibc installation, make some adjustments to ensure that searching and linking take place only within the /tools prefix. Install an adjusted ld, which has a hard-wired search path limited to /tools/lib. Then amend gcc's specs file to point to the new dynamic linker in /tools/lib. This last step is vital to the whole process. As mentioned above, a hard-wired path to a dynamic linker is embedded into every Executable and Link Format (ELF)-shared executable. This can be inspected by running: readelf -l <name of binary> | grep interpreter. Amending gcc's specs file ensures that every program compiled from here through the end of this chapter will use the new dynamic linker in /tools/lib.

The need to use the new dynamic linker is also the reason why the Specs patch is applied for the second pass of GCC. Failure to do so will result in the GCC programs themselves having the name of the dynamic linker from the host system's /lib directory embedded into them, which would defeat the goal of getting away from the host.

During the second pass of Binutils, we are able to utilize the --with-lib-path configure switch to control ld's library search path. From this point onwards, the core toolchain is self-contained and self-hosted. The remainder of the Chapter 5 packages all build against the new Glibc in /tools.

Upon entering the chroot environment in Chapter 6, the first major package to be installed is Glibc, due to its self-sufficient nature mentioned above. Once this Glibc is installed into /usr, perform a quick changeover of the toolchain defaults, then proceed in building the rest of the target HLFS system.