A few days ago I was profiling the startup time of the clang compiler. The callgrind report highlighted that a relatively high amount of time was being spent in the _dl_lookop_symbol_x. A quick gdb inspection of the call stack quickly pointed out that the dynamic linker was spending the time doing dynamic relocations. Tons of them.
objdump -R clang I found out that a whopping 42% of them actually contained the word “clang” in the mangled symbol names. Demangling the names made it clear that they were mostly definitions of the C++ virtual tables for clang internal classes.
Basically 42% of the relocations happening at run time were actually looking for symbols which are obviously defined inside the clang executable itself. So what’s happening here?
Turns out the problem is a poor interaction between the traditional linker (ld, from the binutils package) and C++‘s ODR rule.
ODR stands for One Definition Rule and says that any entity in C++ code must be defined only once. This is obviously trivial for methods, since if they are defined twice, even in different translation units, there will be an error at link time. There are, though, a few things that are implicity defined by the compiler, for example the virtual tables, which are defined whenever a class using virtual methods is declared. (Here I’m speaking informally, I suspect the terminology is not 100% correct) Since the vtable is potentially defined in more than a translation unit, the symbols for the vtable are flagged as weak. Weak symbols will not conflict with each other and they will be all discarded but one (any one is fine). The surviving one will be used by the end product of the linker.
Unfortunately, the compiler does not know if the compiled object file will be used as part of a dynamic library or as part of a main executable. This means that it has to treat the vtable symbols like any library method, since (if the target is a dynamic library) potentially such symbols may be overridden by another definition in a library or in the main executable. This has to happen since the ODR rule must also apply across the library borders for proper support of a few things, especially exception handling.
So at compile time there is no way around using dynamic relocation on the vtable symbols. A possible workaroud would be to compile all the code with the
-fvisibility=hidden flag. Unfortunately this is actually wrong since it may break the ODR rule!
At link time the linker has the chance of eliminating the dynamic relocation, since it does know if the target is a main executable and symbols defined in an executable cannot be overridden by anything (not even by LD_PRELOADed libraries). Unfortunately the traditional ld linker does not apply this optimization.
The new gold linker, originally developed at Google, does tough! It is able to completely eradicate the dynamic relocations to internally defined symbols, effectively reducing the load time.
Moral of the tale: use the gold linker. It should work in most cases (I think kernel code is a notable exception) and generate faster executables while consuming less memory and cpu time during linking.
And please, dear debian/ubuntu maintainers, link clang using gold.