After my previous post about preserved-libs feature, I was asked by many users if I was against this feature because it incremented the usability of Gentoo. I sincerely think I shown before that I don’t want Gentoo to remain for elitists, and thus I won’t be going to say nay to anything to make stuff easier to the users.
On the other hand, I want to make sure that users, excited about being able to upgrade stuff without breaking their system, won’t break it even further. For this reason I want to warn again users against trying the unstable Portage version (2.2) on their stable systems just yet.
Thanks to Zac and Marius, the problem I’m going to talk about is already being taken care of, so it won’t appear once 2.2 final will arrive. So why am I writing about this anyway? Well the reason is actually quite simple: future memory.
On the topic of linking, loaders, ELF files and all the things related, there isn’t a lot of documentation that is available to newcomers. Most of it is very technical and requires good knowledge of many concepts to be useful. I only know of a book that goes in deep of this, Rob Levine’s Linkers and Loaders is probably the most interesting reading on the subject ,as far as I can see. Unfortunately I haven’t tried converting the downloadable version in LRF to load it on the EBook Reader and I haven’t got the printed copy. I should probably get that too, as a reference, I’m just afraid that all the hardcopies I have here will remain at my parents’ house once I move out.
Okay, let’s return back on topic now. What is the problem I found today with preserved-libs? Well as I updated to Xorg 1.5 pre-releases to check if my RandR problems were fixed (by the way, they are), there was a libGL update. I also took the time to switch back to gcc 4.2 for a short time as I wanted to rebuild VirtualBox without ALSA support (whenever I can I’m using PulseAudio), and I could also rebuild xmlrpc-c (the reason why I couldn’t build this with GCC 4.3 is good for a different entry one day). After these builds, Portage told me this:
!!! existing preserved libs:
>>> package: media-libs/mesa-7.1_rc3
* - /usr/lib64/libGLU.so.1.3
* - /usr/lib64/libGLU.so.1.3.070003
>>> package: dev-libs/xmlrpc-c-1.06.27
* - /usr/lib64/libxmlrpc.so.3.6.4
I was surprised because I know libGLU rarely changes interface, and I didn’t think xmlrpc-c would have either. And indeed a quick scanelf run over the files confirmed my ideas:
flame@enterprise ~ % scanelf -S /usr/lib64/libGLU.so*
TYPE SONAME FILE
ET_DYN libGLU.so.1 /usr/lib64/libGLU.so
ET_DYN libGLU.so.1 /usr/lib64/libGLU.so.1
ET_DYN libGLU.so.1 /usr/lib64/libGLU.so.1.3
ET_DYN libGLU.so.1 /usr/lib64/libGLU.so.1.3.070003
ET_DYN libGLU.so.1 /usr/lib64/libGLU.so.1.3.070100
flame@enterprise ~ % scanelf -S /usr/lib64/libxmlrpc.so*
TYPE SONAME FILE
ET_DYN libxmlrpc.so.3 /usr/lib64/libxmlrpc.so
ET_DYN libxmlrpc.so.3 /usr/lib64/libxmlrpc.so.3
ET_DYN libxmlrpc.so.3 /usr/lib64/libxmlrpc.so.3.6.15
ET_DYN libxmlrpc.so.3 /usr/lib64/libxmlrpc.so.3.6.4
As you can see the soname for both the versions of libGLU and both the versions of libxmlrpc is the same. So there is no ABI break, and no good reason for the file to disappear, right?
Indeed if we focus solely on the xmlrpc-c case, the result of preserved-libs actions is that there is now an orphaned unused and unusable shared object:
flame@enterprise ~ % ls -l /usr/lib64/libxmlrpc.so*
lrwxrwxrwx 1 root root 19 2008-07-22 18:04 /usr/lib64/libxmlrpc.so -> libxmlrpc.so.3.6.15
lrwxrwxrwx 1 root root 19 2008-07-22 18:04 /usr/lib64/libxmlrpc.so.3 -> libxmlrpc.so.3.6.15
-rwxr-xr-x 1 root root 56864 2008-07-22 18:04 /usr/lib64/libxmlrpc.so.3.6.15
-rwxr-xr-x 1 root root 56368 2008-02-23 13:36 /usr/lib64/libxmlrpc.so.3.6.4
Let’s see, ld.so (the runtime linker) will look for a library with, as filename, the soname listed in a program’s NEEDED entry in the dynamic seciton of the ELF file. In this case, the soname of libxmlrpc is libxmlrpc.so.3 so that’s what the linker will look for. On my system, that filename is linked to the 3.6.15 version, so the runtime linker will always load the new version of libxmlrpc. The build-time linker (ld) looks instead for the filename with the simple .so prefix (or .a for static libraries). That one is also pointing to 3.6.15.
So who is using the 3.6.4 version that Portage preserved? Well, the only way to use it is to dlopen() it directly and accessing it that way. Which, if anything would be doing that, would be craziness, to the point that I’d rather not see such a software in my system at all. The other option is for an ELF file to have a tweaked NEEDED entry that points explicitly to the full version, but I can tell you I have none of those.
So for now I’ll simply remove the two files, and get rid of the problem. But as you can see, it’s too soon to expect preserve-libs to work out of the box in a perfect manner.
Anyway, Marius, Zac, please keep up with the good work, it’s really needed 🙂 Thanks guys!
[quote]So there is no ABI break, and no good reason for the file to disappear, right?[/quote]You meant no good reason to keep the old version around, right? At least, that’s what your conclusion, that you’re going to simply remove them, indicates.When I read the above, I thought that portage (and I’m using ~arch so the 2.2 rcs, but I too had issues with preserve-libs and now have FEATURES=-preserve-libs set) was “disappearing” (unlinking) libraries that were still needed, presumably both the old and new copies. Keeping around old versions unnecessarily certainly isn’t good, but “disappearing” needed libraries entirely would be far worse, and for a moment that’s what I though was happening based on that wording.Duncan
about the mesa libgl upgrade i have a question:i’ve tried the mesa build that removes 2 .la files (libGL.la and libGLU.la) from /usr/lib/ leaving a lot of rebuild on the system. the problem is that when i try to rebuild the packages that were depending on these files they won’t build and would throw an error about the 2 .la files missing. for what i know the .la files aren’t really needed if not in the case of a static linked system and i was wondering about why they would still be required when rebuilding packages that were depending on them. thanks for the answer.
The solution for the .la files is simply to sed them out:<typo:code>fgrep -rZl /usr/lib/libsomething.la /usr/lib* /usr/kde/*/lib* | xargs -0 sed -i -e ‘s:/usr/lib/libsomething.la::g'</typo:code>For more information about the use of .la files and the reason why removing them is a bit of a mess, please refer to “What about those .la files?”:https://blog.flameeyes.eu/2… and “Again about .la files (or why should they be killed off sooner rather than later)”:https://blog.flameeyes.eu/2… .Duncan, yeah sorry I suppose I should correct the post. Actually, there are very good reasons for removing needed libraries (forcing a revdep-rebuild) like before, rather than leaving cruft around, starting form a security point of view. I’d rather not go in deep how you can make use of a leftover vulnerable library, but it’s not difficult at all…
Diego: Yes, that’s why I decided to disable preserve-libs, at least for now. And, with as-needed in LDFLAGS and a fast enough system, plus routinely running revdep-rebuild after every update session (weekly or more often, deep, newuse), I’ll probably leave it off, as the rebuilds honestly aren’t that big a deal any more.
About the library versions being left. I know that this is not exactly similar, but basically at build time one can specify a specific version although that only really makes sense when you either want to be certain that the old version still works (because of clients with that version), or you want a specific ABI version.The ABI version thing is something our sys-libs/db packages do (and have an eclass to support clients). This allows building with supported versions even when a newer version of the lib is installed. In this case however the numbers are in front of the .so (because of the ld search stripping .so)