Introduction ____________ For my next sets of runs, I wanted to look at various CFLAGS/CXXFLAGS which have been described as 'cheap hardening', i.e. they have a fairly low runtime cost: -D_FORTIFY_SOURCE=2, -fstack-protector-strong, -D_GLIBCXX_ASSERTIONS (the latter only really affects C++, although Fedora also apply it to CFLAGS on the grounds that it will do no harm). These items are additional to tuning-1-packages-and-notes.txt and should be read in conjunction with that (which is where I describe what I build, and how I sometimes have to alter packages to do what I want them to. I have updated the notes in tuning-notes-2.txt and added the new notes to tuning-1-packages-and-notes.txt Package changes since the first set of notes ____________________________________________ In general I have tried to keep the same package versions, but adding patches if necessary. But there have been several higher-profile CVEs which I ought to mention. For the kernel, the tcp_sack problems (initial fix, which turned out to cause soem problems, and subsequent fix) seems a very low risk for a desktop system and I have used the workaround, echo 0 > /proc/sys/net/ipv4/tcp_sac. For firefox, the two CVEs which resulted in 67.0.3 and 67.0.4 needed to be addressed. There were various other changes in 67.0.1 and 67.0.2 that I did not need, so in the end I extracted the CVE fixes and patched 67.0. I did the same on my first build, which I am at the moment retaining for comparisons, but since this uses a lot of rust my timings for repeated builds vary widely. There were similar vulnerabilities fixed in qtwebengine-5.12.4. Although I have been able to drop that over 5.12.3 and qt-5.12.3 on my main systems (saves time) on a fresh system build that does not let falkon get through cmake - it needs a corresponding 5.12.3 version of one of the core Qt libraries, even though it actually links to .so.5 and *runs* ok when that points to 5.12.3. In this case I ripped out all the packages which use Qt (not a lot, for me) from both the original build and the new build, and used5.12.4. For other CVEs I have become aware of, I think that my usage of the test system is at very low risk (e.g. the vlc CVEs announced at the end of June) and I do not plan to change anything in the test builds. My Expectations before I started ________________________________ I had made two assumptions before starting these tests: first, that the only problems from this hardening would be runtime segfaults, and second that my timings for compiles, and for some easily-runnable tests, would increase with the changes. Both assumptions were wrong. I ran three sets of builds, adding the flags in order. It also occurred to me that I would need to test more than I usually do. Unfortunately, one of the things I neglected to test until the final run with all three flags in place was inkscape's fill-bucket tool (and fortifying the source triggers the error there in 0.92.4) - patch now in -patches. Meanwhile, on the first build (just -D_FORTIFY_SOURCE=2) I hit problems: sox hung. In the end, removing that solved the problem (after taking a newer patch from Debian to fix certain vulnerabilities. Testing _______ In LFS, I run all the tests (and use the completed build to repeat the LFS build (up to the end of building bzImage - differences in test results for the repeat get investigated, in one of these runs I had to alter what I was doing for gcc and start over). In BLFS I run very few package tests for my own builds, I'm much more interested in "Does it actually work?". For hardening I've tried to be a bit more extensive in testing some things I barely use. I knew that repeated builds will vary in time, but with the exception of packages which use rustc (and seem to have *very* variable build times) I had assumed that (give or take a tiny percentage of variations) both the build times and the run-times for various runtime uses would either stay the same (e.g. the hardening made no real difference) or increase with each addition. In practice, the "random" variations appeared to outweigh the variations from hardening. So, my runtimes for running an individual test, and sometimes the build times for a package, were often faster when more hardening was used. So, in a non-realtime system the tiem taken for an individual test, or build, can vary more than I had expected. So, I'm not relying on those timings, merely checking that what I was trying to do worked. Instead, I've selected a *few* tests I can easily repeat - documented in the spreadsheet. But I only tried this after adding all three of the cheap hardening options. My current best guess is that for many uses the slowdown from this hardening is less than 2% - often much less. To rephrase that, if my CPU cores were always running at 3600 MHz without the hardening, this is equivalent to underclocking to a bit more than 3500 MHz. However, some of my build times seem to be much more affected, for instance: icu4c with tests 4.4% slower openssh with tests 8.2% slower Python3 rebuild in BLFS 5.9% slower ghostscript with tests 10.2% slower (5.2% of that from -DGLIBCXX_ASSERTIONS) - too me, given the vulnerabilities i nthe last year, that is a price worth paying. gmic (build gimp qt plugin only) 9.6% slower qt-5.12.4 5.7% slower, but still faster than unhardened 5.12.3 :) qtwebengine-5.12.4 2.6% slower texlive source, with tests 17 to 19% slower - this is an example where only adding -D_FORTIFY_SOURCE=2 added 17%, that and strong stack protector 19%, but adding -D_GLIBCXX_ASSERTIONS reduced the increase to 18%. I have not reviewed the times for every package, only for a few, and many of those had build times within 2% of my original figure. Packages which I built where these flags should NOT be used ___________________________________________________________ 1. All packages in chapter 5, there is no point. 2. For gcc, do not use -D_FORTIFY_SOURCE=2 (this matches fedora, and avoids a lot more errors in the testsuite. Use -fstack-protector-strong and -D_GLIBCXX_ASSERTIONS 3. glew - this uses -fno-stack-protector in all its compiles, so upstream clearly do not recommend using it. 4. glibc - do not use any of these, they are unsupported by upstream and will cause breakage. 5. sox - do not use -D_FORTIFY_SOURCE=2, the program will go into a loop. Full disclosure: I was originally NOT setting -D_FORTIFY_SOURCE=2 where a program already used it, but in the end I decided that was not worth the trouble of longer log files. However, on checking now I discovered that I am still doing this for sysvinit, texinfo and openssh. For sysvinit and openssh this is not a problem, they already use it everywhere. But for texinfo it only gets picked up for the perl-related compiles and not for the rest of the compile. I guess that could bite me, if so, I'll find out on my next build. Other packages where things need to be noted ____________________________________________ Inkscape 1. : Fedora had a patch or two for the text tool, but it seemed to work for me and I did not always use it in my testing. I also did not always use the bucket fill tool. In the unhardened version both seem to work. But after supposedly adding all three forms of hardening I tried the bucket fill and inkscape reported an internal error and aborted when I clicked on the dialog. Eventually I found which function was involved, and after following up some google links related to fedora I found one which was specific to this. The problem is an out of bounds [OOB] write (needs -D_FORTIFY_SOURCE=2 to abort) and it has been fixed upstream. The text tool issues were also OOB writes. I've created a patch. Inkscape 2. : Then I looked at hte log from building inkscape and discovered that, uniquely among the C++ cmake packages I build, it was not passing the -D_GLIBCXX_ASSERTIONS flag. I had already come across the sed which Fedora use to remove the FORTIFY from their build (as well as dropping it from their CFLAGS), so I adapted that to instead add this definition. Parole: this has its usual "play mp4, mkv as if they are 5:4 format" issue (no, I have not used a ruler to measure the window, but it looks marginally wider than square. In vlc - and xine, for those mp4/mkv it chooses to play - the aspect ratio is correct. (Qt, etc) Xine (ui and lib) : In my initial build I had not used xine to play mp4 files because it crashed when I tried. But when I was doing extended tests for the hardened packages I forgot about that - it CAN play SOME mp4 and mkv files on SOME builds, other times it segfaults, and sometimes files which have been recoded in current ffmpeg jump back and forth when there is a lot of movement in the scene. The root cause is what ffmpeg reported for one of these source files: [mp4 @ 0x22f56c0] Non-monotonous DTS in output stream 0:1; previous: 7168, current: 6174; changing to 7169. This may result in incorrect timestamps in the output file. When I first noticed this I tried playing an affected rip in ffplay, parole and vlc - none of them had any problems. Summary _______ This exercise relies on tweaking the build processes so that what I specify get used. I have not attempted to look at altering the gcc build to enforce this (or try to - things like -fno-stack-protector and -U_FORTIFY_SOURCE might still override it), so it differs from what was is HLFS. But on modern hardware with desktop programs I think it is worth doing. For a server which is exposed to the public, I think this approach would only be a first step, there are other things (e.g. retpolines in userspace) which might be much more beneficial but also much more costly.