Relationship between –as-needed and –no-undefined – Part 1: what do they do?

I think that after my writeup and Robert’s bugspree some people might have the wrong idea about the relationship between the --as-needed and the --no-undefined flags.

Let’s begin to say exactly what --no-undefined does: it makes the linker reject building targets that have undefined references that are not satisfied by any of the libraries it links to directly and indirectly. The linker already rejects this for final executables, but for a series of reason the default is to allow undefined references in shared objects. But if you have a library A that calls functions from the library B but does not directly link to it, with --no-undefined the linker will refuse to build A entirely; the default would be for A to still build, and the software SW that is going to use A to be forced to explicitly link in B. The following image shows what I mean in the form of graph:

a diagram showing the indirect linking problem

In the above image you can see the “Use relationship” and the “Link relationship” not being balanced, and here comes the problem, since --as-needed has, as task, to remove the link relationships that are not paired with an use relationship. Now just to make sure everybody is on the right page, the reason why --as-needed exists is that, thanks to the original conception of libtool and pkg-config and others, we can easily have programs whose linking diagram is something like the following:

a diagram showing a complex and overextended linking of a program

If you look carefully you can see that there are some linking branches that are not actually used at all; this is because the linker, by default, links in whatever you tell it to link, so you can easily link in a program some libraries that are never ever used, which is not only a waste of time, but it also wastes time and resources at link time since the linker may have to take care of relocations, wastes time during symbol resolution because the extra libraries need to be scanned too, and might take up resources if they have constructor functions for instance, that will cause initialisation functions to be executed, which might open datafiles, allocate data structure and so on.

Now, what we’d like would be something like the following diagram, which shows the linking reduced to the actual needed parts, still following the same rules as the original:

a diagram showing the hopefully pruned out linking of a program

This would reduce the amount of objects loaded to the minimal, needed part, so that no foreign object gets to be loaded at runtime, or linked in at build time for what matters. Unfortunately, this is a dream: the way the linker works, --as-needed produces different results than what you see here, it produces this:

a diagram showing the as-needed pruned linking of a program

I’ve changed colours for the object in this diagram: the yellow objects are the ones that lack a linking relationship, and thus won’t be loaded up, the red objects are those which are broken, since they use an object they don’t link to. You can see that there actually is an exception to that rule, since the yellow object in the middle of the graph uses the blue one at the right end of it, but don’t link directly to it; while it’s probably a good idea to make sure that all you use is also linked in, that situation is legal since the link to the highest object is available indirectly from another object.

Now, --no-undefined would have caught the two broken objects at their build-time rather than when they were to be used by another project, but as I’ll try to explain in the next days, this option is not a panacea, but it helps to identify the issues up in the stream. On the other hand, there are some situation where --as-needed finds trouble that --no-undefined wouldn’t identify early on.

Since people often tell me I write way too much in a single blog entry, I’ll try to wait adding the rest of the content till this entry is digested, the next chapter hopefully in two days, in the mean time feel free to write any question you have in the comments, so I can answer, either on the comments (as you may guess I read them but not always have time to address them directly), or in the next posts.

By the way, the speed to which I can write these article depends directly on the amount of caffeine in my bloodstream, so if you wish to have more content written faster, you can always help me by getting me some good coffee beans, I have never tried java for instance.

3 thoughts on “Relationship between –as-needed and –no-undefined – Part 1: what do they do?

  1. I’d just like to thank you and the others involved in this effort. I think –as-needed is a very usefull flag and have it in my LDFLAGS for years now. In the beginning reporting bugs regarding this sometimes got a very unpleasant receiving, those days matters are much better. Things like the big expat breakage really show how much difference this can make…

    Like

  2. Binutils 2.20 is supposed to actually reproduce the state in the third diagram, unfortunately it doesn’t seem like it’s working quite well right now there is hope for that in the future.

    Like

  3. Hi DiegoThanks for these informative articles, your work is always appreciated.I don’t know if it is connected to the issue you mentioned, but Irrlicht (in portage) has some severe issues with libpng on 64bit platforms. It comes with its own libpng, but that is disabled in the build so it uses the system one instead. And when I try to start my game, I don’t get any textures based on png-images.I filed a bug in gentoo bugzilla (the Irrlicht developers have not been able to reproduce the issue upstream), but I can’t find it any longer. I don’t think anybody looked at it for real.Could this be an issue of linking to the wrong libping, or maybe not linking at all?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s