And here it comes a new post for my technical readers, after a little digression about the Gentoo’s today news, a new post about helping software maintenance by analysis of the resulting binaries.
This time the objective is trying to get rid of functions and variables that are not being used in your codepath. The obvious way to check for this is to use the -Wunused flag during compile, GCC will inform you about unused parameters, local variables and constants, static variables and constants, and static functions. Unfortunately it will not inform you if a non-static function, global variable or constant is not used, as they could be used in a different translation unit.
The linker, though, might know. The linker knows all the translation units that are being linked together, and can analyse what is needed and what is not, to an extent. Unfortunately, it can’t, by default, check for which functions, variables or constants are not used, but we can easily help it out.
First of all, we need the -Wl,--gc-sections option for the linker, but make sure this is not a safe flag to pass globally, so you should never put it in your LDFLAGS in make.conf. What this flag do is to ask the linker to get rid of unused sections. This means for instance that if a file exports a few static constants, and they are never used by anything in the executable you’re building, the linker can drop the .rodata section from that file.
There are a few problems with this, the first problem is that the sections (.data, .rodata, .text, and so on) often contain more than one variable, constant or function, so the linker can’t drop them unless all of them are unused. The other problem is that the the symbols exported by shared objects can be accessed outside the object, and thus can’t be dropped.
The solution to the second problem is to properly use visibility. If the functions are marked with default (or protected) visibility, they can be used externally, so the linker will always think they are used; on the other hand, the functions marked with hidden visibility instead will be checked for being used or not, and could be discarded. So if you want to apply this method on a shared object, the prerequisite is to make sure that only the symbols that are public are being exported.
For the first problem, the solution requires no change to the sources, which means it’s quite easier to take care of, which is good because it affects even executables (non-shared objects). As we said, the linker can only discard an entire section, so the obvious solution is to have every symbol on its own section: every function in a different .text section, very variable in a different .data section and every constant in a different .rodata section. It might sound difficult to do, but the compiler already has two flags that can be used for this: -ffunction-sections and -fdata-sections.
Again: these are not flags you want to use globally. This is especially true since they tend to create way bigger files, which will certainly be slower.
The two flags above will tell the compiler to emit a different section for every symbol, which is exactly what we need: this way the linker will be able to discard single symbols (functions, variables, constants) if they are never referenced.
Now when the linker links the shared object, or the executable, it will take care internally of discarding the unused sections, which will mean unused functions, variables and constants. This is nice, isn’t it? Unfortunately, while you can tweak your build system to use -Wl,--gc-sections whenever available (once you make sure that the sections that will be discarded are fine to be discarded), you shouldn’t force -ffunction-sections and -fdata-sections on users, as the output will be bigger and slower for no good reason.
So you should tweak your source instead of relying on this behaviour. But before tweaking the sources, you need to know what to get rid of. The difficult way to do this is to check the symbols with and without -Wl,--gc-sections, and then remove the symbols not listed in the version with it. This is not what I do 😉 The easy way to do this is to use another option that the GNU linker provides you: -Wl,--print-gc-sections. With this option, all the discarded sections will be reported to you, and so you can easily see what the linker found not to be needed.
ccache x86_64-pc-linux-gnu-gcc -shared .libs/xineplug_decode_gsm610_la-gsm610.o -Wl,--whole-archive ../../contrib/gsm610/.libs/libgsm610.a -Wl,--no-whole-archive -Wl,--rpath -Wl,/home/flame/devel/repos/xine/xine-lib-1.2/enterprise/src/xine-engine/.libs ../../src/xine-engine/.libs/libxine.so -L/usr/lib64 -march=athlon64 -Wl,-z -Wl,defs -Wl,--gc-sections -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--version-script=/home/flame/devel/repos/xine/xine-lib-1.2/enterprise/linker.map -Wl,--print-gc-sections -Wl,-soname -Wl,xineplug_decode_gsm610.so -o .libs/xineplug_decode_gsm610.so
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_add' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_mult' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_mult_r' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_abs' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_L_mult' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_L_add' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.2.2/../../../../x86_64-pc-linux-gnu/bin/ld: Removing unused section '.text.gsm_L_sub' in file '../../contrib/gsm610/.libs/libgsm610.a(add.o)'
[snip]
Code language: JavaScript (javascript)As you can see, this is an actual output from xine-lib building with these option. I set xine-lib-1.2 to use -Wl,--gc-sections whenever available for plugins and xine-lib itself, this because when we use the internal copies of libraries there are already entire sections (without using -ffunction-sections or -fdata-sections) that are unused, and thus using --gc-sections will improve the situation alone already.
The message from the linker should be clear enough, the unused sections will be all listed on stderr, and the name of the section will tell you whether it’s a function, a variable or a constant, as the name of the actual section is prefixed: .text for functions, .data for variables, .rodata for constants. When there’s a number after a variable or constant name, like .5659, that means it’s a local static variable or constant, local to a function. Which function, I’m afraid I do not know how to get offhand.
When I’ll have more time I’ll see if I can get a script to do the work, acting even on non-visibility-enabled shared objects, although that most likely will require a few more performance-crippling gcc flags to be used during build.
We can consider this part of a phase akin to profiling, ran on specially-compiled code, which is not going to be used in production seriously. It can be quite of help though, to identify which part of the code is being maintained and compiled for no reason at all. Or to identify code that should be used and is not.