Home

Papers

Blog

Pictures

About Me

Contact

GitHub


Statically Built Linux: Whece Cometh Our Binaries, There Goeth Our Strength
________________________________________________________________________________

After a light amount of experimentation, it should be noted that crosstool-NG
does exactly all of this +++ more. It's quite exceptional. Check it out. 
To make a static toolchain, just enable Experimental options and musl should
become available. 
$/crosstool-ng/crosstool-ng

Small editorial notes on this:

Section 1 provides some cursory overview of what this project amounts to and the
motivation behind it.

Section 2 will construct a cross-toolchain. If you want to go from one
architecture to another, it's a fine place to start. Just look up what the name
of your target should be, and add a CROSS_COMPILE line to tell musl what you're
up to. musl has famously bad cross-compilation instructions. ymmv.

Section 3 provides a basic fundamental toolchain, and by all accounts you should
be able to start here. Try as I may, however, I was unable to. ymmv.

Section 4 gives you a direction for a final system. You need not accept any of
the choices I have made - indeed, you need not accept any of the choices I made
here at all (except for maybe musl, probably). There's no technically good
reason you couldn't use tcc for most of this, but I reckon you'll run into
issues at some point. clang is an option, but that is far harder to bootstrap,
especially in this context, though not impossible.


Index
________________________________________________________________________________

- Preamble                                                                [0.0]

- Starting Point                                                          [1.0]

- Adapt - Part 1                                                          [2.0]
    - linux-headers                                                       [2.1]
    - binutils                                                            [2.2]
    - gcc - pass 1                                                        [2.3]
    - musl                                                                [2.4]
    - gcc - pass 2                                                        [2.5]

- Evolve - Part 2                                                         [3.0]
    - musl                                                                [3.1]
    - binutils                                                            [3.2]
    - gcc                                                                 [3.3]
    - linux-headers                                                       [3.4]
    - Adjustments                                                         [3.5]
    - libstdcxx                                                           [3.6]
    - m4                                                                  [3.7]
    - bison                                                               [3.8]
    - bzip2                                                               [3.9]
    - busybox                                                             [3.10]
    - zlib                                                                [3.11]
    - gzip (pigz)                                                         [3.12]
    - make                                                                [3.13]
    - xz                                                                  [3.14]

- Overcome - Part 3                                                       [4.0]
    - linux-headers                                                       [4.1]
    - musl                                                                [4.2]
    - Adjustments                                                         [4.3]
    - zlib                                                                [4.4]
    - m4                                                                  [4.5]
    - binutils                                                            [4.6]
    - gcc                                                                 [4.7]
    - bzip2                                                               [4.8]
    - bison                                                               [4.9]
    - flex                                                                [4.10]
    - xz                                                                  [4.11]
    - gzip (pigz)                                                         [4.12]
    - busybox                                                             [4.13]
    - make                                                                [4.14]
    - libressl                                                            [4.15]
    - curl                                                                [4.16]
    - git                                                                 [4.17]

- Conclusion                                                              [5.0]
- References                                                              [6.0]


[0.0] Preamble
________________________________________________________________________________

In my experience, Linux users tend to fall into one of two categories:
pro-static, or anti-static. 

The anti-static crowd seems heavily convinced that statically linking binaries
is ultimately useless, and who could blame them? Perhaps a vast majority of
these inviduals never came up at a time when static linking was relevant (I am
one such person), and their default usage of glibc means that even if they were
to attempt a static linking adventure on their own production machine, they
would be hard pressed to see any benefits in doing so. 

The pro-static community seems to largely overstate the benefits of static
linking. From the few benchmarks I have seen [0], binary size changes seem
poor for the pro-static individual, RAM usage differences seem negligible, and
program launch time seems to bear out few benefits to make it worth the time
investment. Not to mention the absurd difficulty to get any actual information
on how to do static linking on Linux! I leave it for you to search for yourself,
but most of the results merely point to the simple solution: add -static to
$CFLAGS. This solution certainly works for a program you wrote yourself, but if
you attempt to do this for a program with any serious number of dependencies you
won't really have done yourself any favors - ld-libc-$ARCH.so will still attempt
to dynamically load those libraries that weren't statically linked, and you're
SOL. 

Not to mention this 'simple' solution to the question doesn't actually seem to
answer it - just attempt to build curl statically. How would you even begin to
build the latest binutils statically, I wonder? 

However, there are other promising analyses on the subject [1]. Additionally, 
several static-linux projects exist! musl is a testament to at least a
moderate desire to statically link properly, repudiating the poor 'support'
provided by glibc, and projects like oasis [2], sta.li [3], and gales [4]
would seem to proffer up some type of solution to the question: can I run a
statically built Linux system?

All that being said, the power-user buried deep inside me needs to know: can *I*
do it? 

Having been running KISS Linux [5] for a year and going hard-left on its core
principles (being the BDFN of $/dilyn-corner/KISS-kde), I decided to strike
closer to the core of what a more kiss-inspired KISS would endeavor towards.
Additionally, due to the (in general) horrendous documentation on building a
statically linked Linux system from the ground-up, I've decided to author this
work detailing my steps, issues, failures, and success. Hopefully, by the end,
we will have achieved our goal!


[1.0] Starting Point
________________________________________________________________________________

As far as I have been able to divine, this entire enterprise requires more work
than one might initially think - we have to build an actual toolchain before we
can begin this endeavor.

For starters, the -static flag for gcc does exactly one thing: it privileges
static libraries over shared ones during the link step [6]. This would be fine
for, say, some nonfoundational utility where its dependencies also provide their
static libraries in addition to their shared ones (which is not always a given,
mind). However, for our more important and foundational utilities like the ones
provided by binutils, we have to do a bit more work.

If you don't believe me, try it for yourself. Add -static to your CFLAGS, and
build flex. Then try to build binutils; it will fail because it can't find
libfl.so. So we add -static to our LDFLAGS for binutils and... libtool strips it
out. Okay, so add --static instead. Well wouldn't you know it: same error. 
If we build binutils first, we'll be more than able to build flex. But then ar
will fail for any other package we attempt to build, because it wants/needs
libfl! A double bind emerges.
Perhaps just adding --disable-shared to the binutils configure step...
'But it's undocumented!' you cry
'So?' I retort.
$/dilyn-corner/KISS-static/commit/b982f503f027da5e9d14d5ad7d5a111b0e56f5d2

So at the very least, in order to come out of this with a functioning binutils,
we will need to create a static toolchain from the ground-up. What isn't
difficult is identifying how to construct a toolchain; many resources exist
already, and in fact many projects exist that make (cross-)toolchain creation
easy [7]. For now, we can settle for a toolchain guide a la Linux From Scratch
[8]. Because we want to use musl, it might be better if we use Musl-LFS [9] as a
reference. 

Indeed, I ended up following the documentation $/dslm4515 generously provided with
Musl-LFS. While it follows the official LFS documentation quite closely, it 
still provides some useful information. 

The starting point will merely be any musl Linux system - since we're already
building a toolchain from scratch we could start anywhere. For (presumably)
simplicity, we can start from a musl chroot via a KISS tarball. kiss will allow
us to quickly and easily gather sources for our base system, and provide some
ready-to-go patches to fix any musl-related issues we may encounter. Plus, a few
features are already statically built, like busybox. How convenient. 

Because I use wyverkiss [10] as my own system, I considering doing this instead;
however, building llvm statically seems quite challenging, if not at least time
consuming. Not to mention the dependency chain does not yield itself to quick
assembly - cmake and python? Too much too soon.  It's not impossible, mind:
$/dilyn-corner/KISS-me/blob/wyverkiss-static/core/llvm/build


[2.0] Adapt - Part 1
________________________________________________________________________________

I spent a large amount of time building and troubleshooting during this project
- almost enough to rival my first LFS setup. I ran into numerous issues, mostly
related to GCC erroring out during the build at various times. At first, I
attempted to simply start from creating the basic tools. However, no matter
which ways I configured and compiled binutils, gcc, and musl, I would always
arrive at some build failure at some stage of the toolchain creation. Out of
frustration, I settled for building an initial 'cross' toolchain - scare quotes
because there is nothing cross about the toolchain, architecture-wise. Of
course, that failed as well. UNTIL:

$HOST=x86_64-pc-linux-musl -> $HOST=x86_64-kiss-linux-musl

I have absolutely no idea why this worked. But switching $TARGET, $HOST, and
$BUILD to this new variable (different from the $(gcc -dumpmachine) which
vanilla KISS would output) resulted in, get this: zero. errors. 

Hallelujah. Two days of tedious configuring, compiling, and source code reading
end by a simple sed. If there were a God, he would be laughing.

The first part of our cross-toolchain will consist of the usual suspects:
binutils, gcc, libc, kernel headers. For this initial toolchain,
CFLAGS=CXXFLAGS= . Disabling shared libraries is also something we should look 
to do, that way when we eventually construct our toolchain-proper we won't have
any shared libraries available to our system. Ideally, everything in this
portion should link to the host's libc.so. 

If you want to be safe, make a user that doesn't have write access to /usr. This
way you don't accidentally overwrite something important.

Because we're creating a cross-toolchain, you could theoretically build for an
ARM system. I haven't tested this, nor do I have any interest in doing so. ymmv.

Important environment variables to set and things to do:

mkdir -pv /mnt/static/tools 
mkdir -pv /mnt/static/sources
mkdir -pv /mnt/static/cross-tools 

ln -sv /mnt/static/tools       /tools
ln -sv /mnt/static/cross-tools /cross-tools

export CFLAGS=
export CXXFLAGS=
export KISS_ARCH=x86
export KISS_CPU=x86-64 
export STATIC=/mnt/static
export MAKEFLAGS=-j$(nproc)
export KISS_HOST=$(gcc -dumpmachine)
export KISS_TARGET=x86_64-kiss-linux-musl
export PATH=/cross-tools/bin:/tools/bin:/usr/bin

Thus begineth the building.

    [2.1] linux-headers
    ____________________________________________________________________________
    
    We need some headers as these will form the basis of our system.
    
    tar xvf linux-*.tar.xz
    cd linux-*
    
    make mrproper
    
    make ARCH=${KISS_ARCH} headers_check
    make ARCH=${KISS_ARCH} headers
    
    mkdir -pv /cross-tools/${KISS_TARGET}/include
    cp    -rv usr/include/* /cross-tools/${KISS_TARGET}/include
    rm    -v  /cross-tools/${KISS_TARGET}/include/Makefile
    
    cd ..
    rm -rvf linux-*
    
    
    [2.2] binutils
    ____________________________________________________________________________
    
    binutils provides the basic utilities we need to actually assemble 
    everything. Additionally, utilizing nm and ar we can identify where any 
    issues exist - I never saw any that prompted the need, but they're there.
    
    tar xvf binutils-*.tar.xz
    cd binutils-*
    mkdir -v build && cd build
    
    ../configure \
        --prefix=/cross-tools \
        --target=${KISS_TARGET} \
        --with-sysroot=/cross-tools/${KISS_TARGET} \
        --disable-nls \
        --disable-shared \
        --disable-werror \
        --disable-multilib
    
    make configure-host
    make
    make install
    
    cd ../..
    rm -rvf binutils-*
    
    
    [2.3] gcc - pass 1
    ____________________________________________________________________________
    
    What's important to note about gcc is that if host==target==build, gcc will
    bootstrap itself. If they're different, bootstrapping is disabled. We just
    need a basic, barebones gcc at this stage. Luckily, that fact saves us some
    time here.
    
    tar xvf gcc-*.tar.xz
    cd gcc-*
    
    tar xvf ../gmp-* 
    tar xvf ../mpfr-*
    tar xvf ../mpc-*
    
    mv gmp-*  gmp
    mv mpfr-* mpfr
    mv mpc-*  mpc
    
    mkdir -v build && cd build
    
    ../configure \
        --prefix=/cross-tools \
        --host=${KISS_HOST} \
        --build=${KISS_HOST} \
        --target=${KISS_TARGET} \
        --with-newlib \
        --with-arch=${KISS_CPU} \
        --without-headers \
        --disable-nls \
        --disable-libitm \
        --disable-libvtv \
        --disable-libssp \
        --disable-shared \
        --disable-libgomp \
        --disable-threads \
        --disable-multilib \
        --disable-libatomic \
        --disable-libstdcxx \
        --disable-libquadmath \
        --disable-libsanitizer \
        --disable-decimal-float \
        --enable-languages=c \
        --enable-clocale=generic
    
    make all-gcc all-target-libgcc
    make install-gcc install-target-libgcc
    
    cd ../..
    rm -rvf gcc-*
    
    
    [2.4] musl
    ____________________________________________________________________________
    
    musl will provide us some extra headers we need, not to mention the all
    important libc.a. If we were cross-compiling, now would be the important 
    time to inform musl of this fact.
    
    tar xvf musl-*.tar.gz
    cd musl-*
    
    ./configure \
        --prefix=/ \
        --target=${KISS_TARGET} \
        --disable-shared
    
    make
    make DESTDIR=/cross-tools install
    
    mkdir -v /cross-tools/usr
    ln -sv ../include /cross-tools/usr/include
    
    cd ..
    rm -rvf musl-*
    
    
    [2.5] gcc - pass 2
    ____________________________________________________________________________
    
    We build gcc again here to add in some extra stuff, make some tweaks, and to
    feel better about ourselves.
    
    tar xvf gcc-*.tar.xz
    cd gcc-*
    
    tar xvf ../gmp-* 
    tar xvf ../mpfr-*
    tar xvf ../mpc-*
    
    mv gmp-*  gmp
    mv mpfr-* mpfr
    mv mpc-*  mpc
    
    sed -i '/m64=/s/lib64/lib/' gcc/config/i386/t-linux64
    sed -i 's/lib64/lib/'       gcc/config/i386/linux64.h
    
    mkdir -v build && cd build
    
    AR=ar LDFLAGS="-Wl,-rpath,/cross-tools/lib" \
    ../configure \
        --prefix=/cross-tools \
        --host=${KISS_HOST} \
        --build=${KISS_HOST} \
        --target=${KISS_TARGET} \
        --with-sysroot=/cross-tools \
        --disable-nls \
        --disable-shared \
        --disable-libssp \
        --disable-multilib \
        --disable-lto-plugin \
        --disable-libsanitizer \
        --disable-libstdcxx-pch \
        --enable-threads=posix \
        --enable-clocale=generic \
        --enable-languages=c,c++
    
    make AS_FOR_TARGET="${KISS_TARGET}-as" \
         LD_FOR_TARGET="${KISS_TARGET}-ld"
    
    make install
    
    cd ../..
    rm -rvf gcc-*
    
    
    Delete some .la files that we don't want or need
    find /cross-tools -name '*.la' -exec rm -fv {} \;
    
    Ultimately, running ldd on any of the binaries we have here should output
    that they are only linked against the host libc.so, with the exception of
    ar and ranlib which require libfl (flex). Additionally, there should only
    be one shared library on the system; libgcc1.so. In the next section these
    requirements shall fall away.
    
    
[3.0] Evolve - Part 2 
________________________________________________________________________________

Our primary goal in this section is to take our minimally-built toolchain and
expand it into a more proper one with all the static libraries we would need.
This process will be relatively similar to the previous one, there are just more
things to build here.

As I said at the very beginning, you should technically be able to start here.
All you should technically have to do is CFLAGS="-static -static-libcc
-static-libstdc++", and do two passes - the first pass to make sure you have all
the static libraries with the below programs (in addition to the shared
counterparts to ensure things work), and the second to link only against the
static libraries. At least, that's what I would (perhaps naively) think should
work.


    [3.1] musl
    ____________________________________________________________________________
    
    Here we build musl again just to put it in /tools. If we were 
    cross-compiling, we would continue to do so here.
    
    tar xvf musl-*
    cd musl-*
    
    ./configure \
        --prefix=/ \
        --target=${KISS_TARGET} \
        --disable-shared
    
    make
    make DESTDIR=/tools install
    
    
    Adjustments
    ____________________________________________________________________________
    
    Out of an overabundance of caution, we should take care to adjust our 
    toolchain so that everything points to the correct things. Technically I 
    don't think that this is necessary for our purposes, but it is worth 
    documenting (and ultimately harmless if we do it right).
    
    export tmpspec=`dirname $(${KISS_TARGET}-gcc -print-libgcc-file-name)`/specs
    ${KISS_TARGET}-gcc -dumpspecs > specs
    
    Modify dumped specs file for every instance of:
    sed -i 's/\/lib\/ld-musl-x86_64.so.1/\/tools\/lib\/ld-musl-x86_64.so.1/g' \
        specs
    
    Check:
    grep "/tools/lib/ld-musl-x86_64.so.1" specs
    
    Install modified specs to the cross toolchain:
    mv -v specs $tmpspec
    unset tmpspec
    
    Quick check the toolchain:
    echo 'int main(){}' > dummy.c
    ${KISS_TARGET}-gcc dummy.c
    ${KISS_TARGET}-readelf -l a.out | grep Requesting
    
    Note that for a static file the last line won't return anything. We're 
    mostly just ensuring that ${KISS_TARGET}-gcc can at least build something.
    
    At this point, we want to make sure that every instance of our main tools 
    (cc, ar, etc) all point to the correct objects:
    
    export CC="${KISS_TARGET}-gcc"
    export CXX="${KISS_TARGET}-g++"
    export AR="${KISS_TARGET}-ar"
    export AS="${KISS_TARGET}-as"
    export RANLIB="${KISS_TARGET}-ranlib"
    export LD="${KISS_TARGET}-ld"
    export STRIP="${KISS_TARGET}-strip"
    
    For legacy purposes:
    ln -sv lib /tools/lib64
    
    
    [3.2] binutils
    ____________________________________________________________________________
    
    tar xvf binutils-*.tar.xz
    cd binutils-*
    
    mkdir -v build && cd build
    
    ../configure \
        --prefix=/tools \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET} \
        --target=${KISS_TARGET} \
        --disable-nls \
        --disable-shared \
        --disable-werror \
        --with-sysroot \
        --with-lib-path=/tools/lib
    
    make
    make install
    
    Make a special ld to search the canonical place:
    make -C ld clean
    make -C ld LIB_PATH=/usr/lib:/lib
    cp -v ld/ld-new /tools/bin
    
    We will utilize this ld in building the final system.
    
    cd ../..
    rm -rvf binutils-*
    
    
    [3.3] gcc
    ____________________________________________________________________________
    
    tar xvf gcc-*.tar.xz
    cd gcc-*
    
    tar xvf ../gmp-* 
    tar xvf ../mpfr-*
    tar xvf ../mpc-*
    
    mv gmp-*  gmp
    mv mpfr-* mpfr
    mv mpc-*  mpc
    
    cd gcc/config
    for file in linux.h i386/linux.h i386/linux64.h
    do
      sed -i 's@/lib\(64\)\?\(32\)\?/ld@/tools&@g' $file
      sed -i 's@/usr@/tools@g' $file
      echo '
    #undef STANDARD_STARTFILE_PREFIX_1
    #undef STANDARD_STARTFILE_PREFIX_2
    #define STANDARD_STARTFILE_PREFIX_1 "/tools/lib/"
    #define STANDARD_STARTFILE_PREFIX_2 ""' >> $file
    done
    
    cd ../..

    mkdir -v build && cd build
    
    CFLAGS=-static \
    ../configure \
        --target=${KISS_TARGET} \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET} \
        --prefix=/tools \
        --with-local-prefix=/tools \
        --with-native-system-header-dir=/tools/include \
        --disable-libitm \
        --disable-libssp \
        --disable-libvtv \
        --disable-shared \
        --disable-libgomp \
        --disable-multilib \
        --disable-bootstrap \
        --disable-libquadmath \
        --disable-libsanitizer \
        --disable-libstdcxx-pch \
        --enable-languages=c,c++
    
    PATH=/bin:/usr/bin:/cross-tools/bin:/tools/bin make
    make install
    
    ln -sv gcc /tools/bin/cc 
    
    INCLUDEDIR=`dirname $(${KISS_TARGET}-gcc -print-libgcc-file-name)`/include
    find ${INCLUDEDIR}/* -maxdepth 0 -type d -exec rm -rvf '{}' \;
    rm -vf `grep -l "DO NOT EDIT THIS FILE" ${INCLUDEDIR}/*`
    unset INCLUDEDIR
    
    cd ../..
    rm -rvf gcc-*
    
    
    [3.4] linux-headers
    ____________________________________________________________________________
    
    tar xvf linux-*.tar.xz
    
    cd linux-*
    
    make mrproper
    make ARCH=${KISS_ARCH} headers
    
    cp -rv usr/include/* /tools/include
    rm -v /tools/include/Makefile
    
    cd ..
    rm -rvf linux-*
    
    
    [3.5] Adjustments
    ____________________________________________________________________________
    
    Once again, we can adjust our toolchain to point to the proper location. 
    Again, this is not totally necessary, but is worth documenting in general.
    
    # Dump gcc specs
    SPECFILE=`dirname $(${KISS_TARGET}-gcc -print-libgcc-file-name)`/specs &&
    ${KISS_TARGET}-gcc -dumpspecs > tempspecfile
    
    # Modify dumped tempspecfile file for every instance of:
    # /lib/ld-musl-$ARCH.so.1 to /tools/lib/ld-musl-$ARCH.so.1 
    
    sed -i 's/\/lib\/ld-musl-x86_64.so.1/\/tools\/lib\/ld-musl-x86_64.so.1/g' \
        tempspecfile

    Check:
    grep "/tools/lib/ld-musl-x86_64.so.1" tempspecfile
    
    mv -vf tempspecfile $SPECFILE &&
    unset SPECFILE
    
    INCLUDEDIR=`dirname $(${KISS_TARGET}-gcc -print-libgcc-file-name)`/include
    find ${INCLUDEDIR}/* -maxdepth 0 -type d -exec rm -rvf '{}' \;
    rm -vf `grep -l "DO NOT EDIT THIS FILE" ${INCLUDEDIR}/*`
    unset INCLUDEDIR
    
    Finally, a sanity check to make sure we didn't break anything. The last line
    should output nothing if it's static.
    
    echo 'int main(){}' > dummy.c
    ${KISS_TARGET}-gcc -B/tools/lib dummy.c
    readelf -l a.out | grep ': /tools'
    
    
    [3.6] libstdcxx
    ____________________________________________________________________________
    
    tar xvf gcc-*.tar.xz
    cd gcc-*

    mkdir -v build && cd build
    
    ../libstdc\+\+-v3/configure \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET} \
        --target=${KISS_TARGET} \
        --prefix=/tools \
        --disable-nls \
        --disable-shared \
        --disable-multilib \
        --disable-libstdcxx-pch \
        --disable-libstdcxx-threads \
        --with-gxx-include-dir=/tools/${KISS_TARGET}/include/c++/10.2.0
    
    make
    make install
    
    cd ../..
    rm -rvf gcc-*
    
    
    [3.7] m4
    ____________________________________________________________________________
    
    tar xvf m4-*.tar.xz
    cd m4-*
    
    ./configure \
        --prefix=/tools \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET}
    
    make
    make install
    
    cd ..
    rm -rvf m4-*
    
    
    [3.8] bison
    ____________________________________________________________________________
    
    tar xvf bison-*.tar.xz
    cd bison-*
    
    ./configure \
        --prefix=/tools \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET}
    
    make
    make install
    
    cd ..
    rm -rvf bison-*
    
    
    [3.9] bzip2
    ____________________________________________________________________________
    
    tar xvf bzip2-*.tar.gz
    cd bzip2-*
    
    make CC="cc $CFLAGS -static"
    
    for bin in bzip2 bzdiff bzgrep bzip2recover bzmore; do
        install -Dm755 "$bin" "/tools/bin/$bin"
    done
    
    install -Dm644 libbz2.a /tools/lib/libbz2.a
    install -Dm644 bzlib.h  /tools/include/bzlib.h
    
    cd ..
    rm -rvf bzip-*
    
    
    [3.10] busybox
    ____________________________________________________________________________
    
    Refer to the KISS package [11]. Ensure it is installed to /tools/bin.
    
    
    [3.11] zlib
    ____________________________________________________________________________
    
    tar xvf zlib-*.tar.gz
    cd zlib-*
    
    ./configure \
        --prefix=/tools \
        --static
    
    make
    make install
    
    cd ..
    rm -rvf zlib-*
    
    
    [3.12] gzip (pigz)
    ____________________________________________________________________________
    
    tar xvf pigz-*.tar.gz
    cd pigz-*
    
    make CC="${CC:-cc} -static" \
        CPPFLAGS=-I/tools/include \
        LDFLAGS=-L/tools/lib
    
    install -Dm755 pigz   /tools/bin/pigz
    install -Dm755 unpigz /tools/bin/unpigz
    
    ln -sv pigz   /tools/bin/gzip
    ln -sv pigz   /tools/bin/zcat
    ln -sv unpigz /tools/bin/gunzip
    
    cd ..
    rm -rvf pigz-*
    
    
    [3.13] make
    ____________________________________________________________________________
    
    tar xvf make-*.tar.gz
    cd make-*
    
    ./configure \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET} \
        --prefix=/tools \
        --disable-nls
    
    make
    make install
    
    cd ..
    rm -rvf make-*
    
    
    [3.14] xz
    ____________________________________________________________________________
    
    tar xvf xz-*.tar.bz2
    cd xz-*
    
    ./configure \
        --build=${KISS_HOST} \
        --host=${KISS_TARGET} \
        --prefix=/tools \
        --disable-nls \
        --disable-shared
    
    make
    make install
    
    cd ..
    rm -rvf xz-*


    # Remove garbage .la files:
    find /tools -name '*.la' -exec rm -f {} \;
    
    If you've done everything right, ldd should report that all of the binaries 
    in /tools/bin are not valid dynamic programs. Additionally, /tools/lib 
    should only contain *.a and crt*.o files. You could strip these things if 
    you wanted to, but does it really matter? It's useful if you plan on keeping 
    these things around as a backup in case something goes wrong.
    
    strip --strip-debug /tools/lib/*
    /usr/bin/strip --strip-unneeded /tools/bin/*
    
    # Remove the documentation:
    rm -rf /tools/share/info
    rm -rf /tools/share/man
    rm -rf /tools/share/doc
    

[4.0] Overcome - Part 3
________________________________________________________________________________

Now that we have an assemblage of static tools and libraries, we can go ahead
and build up a root system in the usual way, with only minor tweaks to ensure
things are found and linked properly.

As new tools come in, we want to make sure they are given priority over our
previous tools. We can employ a chroot to make things easier on ourselves. And
Finally, add -static -static-libgcc -static-libstdc++ to CFLAGS... just in case.

You could also use the kiss-chroot script [13] with some small modifications so 
that the virtual filesystems are mounted properly. In fact, I would suggest it. 
I won't go over mounting the relevant /dev, /proc, /sys stuff here.

chroot "$STATIC" /tools/bin/env -i \
    HOME=/root \
    TERM="$TERM" \
    PS1='(static) $PWD -> ' \
    PATH=/bin:/tools/bin \
    /tools/bin/sh

Next we can setup the filesystem hierarchy if that's what we're going for. 
I would recommend installing baselayout [14] from KISS just to make this easier.

Remember: the following are run from within a chroot - adjust as needed if you
aren't chrooted. Otherwise, you might break something.


    [4.1] linux-headers
    ____________________________________________________________________________
    
    tar xvf linux-*.tar.xz
    cd linux-*
    
    make mrproper
    make headers
    
    # Theoretically dangerous. I live on the edge.
    find usr/include -name '.*' -exec rm -f {} \;
    
    cp -rv usr/include/*  /usr/include
    rm -v /usr/include/Makefile
    
    cd ..
    rm -rvf linux-*
    
    
    [4.2] musl
    ____________________________________________________________________________
    
    tar xvf musl-*.tar.gz
    cd musl-*
    
    ./configure \
        --prefix=/usr \
        --disable-shared
    
    make
    make install
    
    cd ..
    rm -rvf musl-* 
    
    
    [4.3] Adjustments
    ____________________________________________________________________________
    
    We've gotta tighten up our toolchain one last time. 

    Our target should match $(gcc -dumpmachine)
    export KISS_TARGET=x86_64-kiss-linux-musl
    
    mv -v  /tools/bin/ld                /tools/bin/ld-old
    mv -v  /tools/${KISS_TARGET}/bin/ld /tools/${KISS_TARGET}/bin/ld-old
    mv -v  /tools/bin/ld-new            /tools/bin/ld
    ln -sv /tools/bin/ld                /tools/${KISS_TARGET}/bin/ld
    
    export SPECFILE=`dirname $(gcc -print-libgcc-file-name)`/specs
    gcc -dumpspecs | sed -e 's@/tools@@g'                   \
        -e '/\*startfile_prefix_spec:/{n;s@.*@/usr/lib/ @}' \
        -e '/\*cpp:/{n;s@$@ -isystem /usr/include@}' >  tempspecfile
    
    Inspect the file to ensure that every occurance of 
    /tools/lib/ld-musl-$ARCH.so.1 was changed to /lib/ld-musl-$ARCH.so.1, 
    and that the following lines appear:
    
    *startfile_prefix_spec:
    /usr/lib
    
    *cpp:
    %{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT} -isystem /usr/include
    
    Now just make our tweaks permanent!

    mv -vf tempspecfile $SPECFILE &&
    unset SPECFILE  KISS_TARGET
    
    Finally, ensure we did not break our compiler:
    echo 'int main(){}' > dummy.c
    cc dummy.c -v -Wl,--verbose &> dummy.log
    readelf -l a.out | grep ': /lib'
    
    Make sure we're using the right start files:
    grep 'crt1.o succeeded' dummy.log
    grep 'crti.o succeeded' dummy.log
    grep 'crtn.o succeeded' dummy.log
    Should output:
    /usr/lib/crt1.o succeeded
    /usr/lib/crti.o succeeded
    /usr/lib/crtn.o succeeded
    
    And to ensure that our compiler searches the right directory,
    grep -B1 ' ^ /usr/include' dummy.log
    Should output
    #include <...> search starts here
    /usr/include
    
    Finally, make sure that our linker looks in the right places:
    grep 'SEARCH.*/usr/lib' dummy.log
    Should output:
    SEARCH_DIR("=/tools/x86_64-kiss-linux-musl/lib64")
    SEARCH_DIR("/usr/lib")
    SEARCH_DIR("/lib")
    SEARCH_DIR("=/tools/x86_64-kiss-linux-musl/lib")
    
    If everything so far is right, then nothing is broken. If something is off, 
    go back to the relevant step and you should hopefully be able to recover.
    
    
    [4.4] zlib
    ____________________________________________________________________________
    
    tar xvf zlib-*.tar.gz
    cd zlib-*
    
    ./configure \
        --prefix=/usr \
        --libdir=/usr/lib \
        --static
    
    make
    make install
    
    cd ..
    rm -rvf zlib-*
    
    
    [4.5] m4
    ____________________________________________________________________________
    
    tar xvf m4-*.tar.xz
    cd m4-*
    
    ./configure \
        --prefix=/usr 
    
    make
    make install
    
    cd ..
    rm -rvf m4-*
    
    
    [4.6] binutils
    ____________________________________________________________________________
    
    Here we specify the include and lib dirs, lest binutils protest too much.
    
    tar xvf binutils-*.tar.xz
    cd binutils-*
    
    mkdir -v build && cd build
    
    
    CPPFLAGS="-I/tools/include" \
    LDFLAGS="-L/tools/lib" \
    ../configure \
        --prefix=/usr \
        --disable-gdb \
        --disable-nls \
        --disable-gold \
        --disable-gprof \
        --disable-werror \
        --disable-multilib \
        --disable-readline \
        --enable-lto \
        --enable-plugins \
        --enable-ld=default \
        --with-mmap \
        --with-system-zlib \
        --with-targets=x86_64-pep \
        --with-static-standard-libraries \
        --with-lib-path=/usr/lib:/usr/local/lib
    
    make configure-host
    make 
    make install
    
    cd ../..
    rm -rvf binutils-*
    
    
    [4.7] gcc
    ____________________________________________________________________________
    
    Much like binutils, some specifications must be made.
    
    tar xvf gcc-*.tar.xz
    cd gcc-*
    
    tar xvf ../gmp-*
    tar xvf ../mpfr-*
    tar xvf ../mpc-*
    
    mv gmp-*  gmp
    mv mpfr-* mpfr
    mv mpc-*  mpc
    
    sed -i '/m64=/s/lib64/lib/' gcc/config/i386/t-linux64
    sed -i 's/lib64/lib/'       gcc/config/i386/linux64.h
    
    mkdir -v build && cd build
    
    CPPFLAGS="-I/tools/include" \
    LDFLAGS="-L/tools/lib" \
    ../configure \
        --prefix=/usr \
        --libexecdir=/usr/lib \
        --mandir=/usr/share/man \
        --infodir=/usr/share/info \
        --build=x86_64-pc-linux-musl \
        --disable-nls \
        --disable-libmpx \
        --disable-shared \
        --disable-werror \
        --disable-multilib \
        --disable-bootstrap \
        --disable-libmudflap \
        --disable-libsanitizer \
        --disable-fixed-point \
        --disable-libstdcxx-pch \
        --enable-tls \
        --enable-threads \
        --enable-default-ssp \
        --enable-__cxa_atexit \
        --enable-languages=c,c++ \
        --enable-checking=release \
        --without-included-gettext \
        --with-system-zlib
    
    make
    make install
    
    cd ../..
    rm -rvf gcc-*
    
    
    [4.8] bzip2
    ____________________________________________________________________________
    
    tar xvf bzip2-*.tar.gz
    cd bzip2-*
    
    make CC="${CC:-cc} $CFLAGS"
    for bin in bzip2 bzdiff bzgrep bzip2recover bzmore; do
        install -Dm755 $bin /usr/bin/$bin
    done
    
    install -Dm644 libbz2.a /usr/lib/libz2.a
    install -Dm644 bzlib.h  /usr/include/bzlib.h
    
    cd ..
    rm -rvf bzip2-*
    
    
    [4.9] bison
    ____________________________________________________________________________
    
    tar xvf bison-*.tar.xz
    cd bison-*
    
    ./configure \
        --prefix=/usr \
        --disable-nls \
        --without-libtextstyle-prefix
    
    make
    make install
    
    cd ..
    rm -rvf bison-*
    
    
    [4.10] flex
    ____________________________________________________________________________
    
    tar xvf flex-*.tar.gz
    cd flex-*
    
    ./configure \
        --prefix=/usr \
        ac_cv_func_malloc_0_nonnull=yes \
        ac_cv_func_realloc_0_nonnull=yes \
        --disable-nls \
        --disable-shared
    
    make
    make install
    
    cd ..
    rm -rvf flex-*
    
    
    [4.11] xz
    ____________________________________________________________________________
    
    tar xvf xz-*.bz2
    cd xz-*
    
    ./configure \
        --prefix=/usr \
        --disable-nls \
        --disable-shared
    
    make
    make install
    
    cd ..
    rm -rvf xz-*
    
    
    [4.12] gzip (pigz)
    ____________________________________________________________________________
    
    tar xvf pigz-*.tar.gz
    cd pigz-*
    
    make CC="${CC:-cc} $CFLAGS"
    
    install -Dm755 pigz /usr/bin/pigz
    install -Dm755 unpigz /usr/bin/unpigz
    
    ln -sv pigz /usr/bin/gzip
    ln -sv pigz /usr/bin/zcat
    ln -sv unpigz /usr/gin/gunzip
    
    cd ..
    rm -rvf pigz-*
    
    
    [4.13] busybox
    ____________________________________________________________________________
    
    The KISS build is already statically built; use that [11]!
    
    
    [4.14] make
    ____________________________________________________________________________
    
    tar xvf make-*.tar.gz
    cd make-*
    
    ./configure \
        --prefix=/usr \
        --disable-nls
    
    make
    make install
    
    cd ..
    rm -rvf make-*
    
    
    [4.15] libressl
    ____________________________________________________________________________
    
    tar xvf libressl-*.tar.gz
    cd libressl-*
    
    ./configure \
        --prefix=/usr \
        --sysconfdir=/etc \
        --disable-shared \
        --disable-tests
    
    make
    make install
    
    cd ..
    rm -rvf libressl-*
    
    
    [4.16] curl
    ____________________________________________________________________________
    
    KISS normally deletes .la files for us to evade this problem, but we aren't
    using KISS. so do it here, lest curl complains.
    
    find /usr/lib -name '*.la' -exec rm -f {} \;
    
    tar xvf curl-*.tar.xz
    cd curl-*
    
    ./configure \
        --prefix=/usr \
        --enable-ipv6 \
        --enable-unix-sockets \
        --enable-symbol-hiding \
        --disable-manual \
        --disable-ldap \
        --disable-ares \
        --disable-rtmp \
        --disable-shared \
        --without-icu \
        --without-libpsl \
        --without-libidn \
        --without-libidn2 \
        --with-ca-fallback
    
    make curl_LDFLAGS=-all-static
    make install
    
    cd ..
    rm -rvf curl-*
    
    
    [4.17] git
    ____________________________________________________________________________
    
    cat > config.mak << EOF
    NO_GETTEXT=YesPlease
    NO_SVN_TESTS=YesPlease
    NO_TCLTK=YesPlease
    NO_EXPAT=YesPlease
    NO_NSEC=YesPlease
    NO_PYTHON=YesPlease
    NO_PERL=YesPlease
    NO_SVN_TESTS=YesPlease
    NO_SYS_POLL_H=1
    NO_CROSS_DIRECTORY_HARDLINKS=1
    NO_INSTALL_HARDLINKS=1
    EOF
    
    ./configure \
        --prefix=/usr \
        --libexecdir=/usr/lib \
        ac_cv_lib_curl_curl_global_init=yes \
        ac_cv_snprintf_returns_bogus=no \
        ac_cv_fread_reads_directories=yes
    
    make LIBS="$(curl-config --static-libs) libgit.a xdiff/lib.a -lz"
    make install


[5.0] Conclusion
________________________________________________________________________________

And with that, we have what basically amounts to a KISS system. 

Before you start deleting /usr/lib, however, might I recommend actually building
the system you want? Unless you just want a simple backup system or something,
this collection of utilities won't do you a whole lot. Unless you merely want to
interact with git and wget your webpages, that is. 

This exercise was mostly just that: an exercise. An attempt at discovering
whether I, a novice user, could construct a system of old. It isn't terribly
difficult - no real harder than a LFS system - but it certainly took time. I
spent a few days reading documentation, tweaking configure options, and
refreshing my knowledge in general on toolchain construction. And then I of
course deleted the lib dir (and the backup lib dir!), and realized I had to
start over - gcc was broken. Moral of the story: use kiss and rebuild your
system once more, just to be safe. 


If you're interested in following this project, the outcome will exist on
$/dilyn-corner/KISS-static and will host a basic tarball (the one we've built
here), along with the official KISS repositories adapted for our needs. 


[6.0] References
________________________________________________________________________________

[0]  https://etalabs.net/compare_libcs
[1]  https://drewdevault.com/dynlib
[2]  https://github.com/oasislinux/oasis
[3]  https://sta.li
[4]  https://fixpoint.welshcomputing.com/2019/introducing-gales-linux-a-cross-bootstrappeddo-t-yourself-fully-static-discriminatory-distribution
[5]  https://stackoverflow.com/questions/8692128/static-option-for-gcc
[6]  https://k1ss.org
[7]  https://github.com/crosstool-ng/crosstool-ng
[8]  https://linuxfromscratch.org
[9]  https://github.com/dslm4515/Musl-LFS
[10] https://github.com/wyvertux/wyverkiss
[11] https://github.com/kisslinux/repo/tree/master/core/busybox
[12] https://github.com/kisslinux/repo/tree/master/core/git
[13] https://github.com/kisslinux/kiss/blob/master/contrib/kiss-chroot
[14] https://github.com/kisslinux/repo/tree/master/core/baselayout


________________________________________________________________________________

Dilyn Corner (C) 2020-2022