One thing that might be interested to start doing in Gentoo to improve the users’ experience is to start playing with hidden visibility.
While I’ve already written before that hidden visibility is risky for most libraries unless upstream predisposed them to be used with hidden visibility (xine, most recent GNU libraries excluding glibc), it is usually a no-problem for final executables, as they aren’t supposed to export symbols at all, most of the times at least (there are some cases where binaries export symbols that they want to override from other libraries, like the C library or stuff like that).
Users should not play with -fvisibility=hidden flag as that might break badly; but we developers should know better. It woudl be nice to check if we’re using GCC 4.0 or later (you don’t want to use hidden visibility with older compilers), and then append the flag, for software we know is compatible with this. This software usually includes final executables (no shared objects) and static libraries (which usually should be linked without exporting the symbols, as otherwise you might end up breaking stuff badly, if two shared objects link two different versions of the same static library).
By using hidden visibility we should be able to reduce the amount of possible symbols’ collisions, we might be able to reduce the size of the applications, and finally we reduce startup, as the binary’s symbols are not resolved at runtime by the dynamic linker.
This combined with --gc-sections should also mitigate the problems caused by mishandled commodity libraries. The sad note is that using this my script won’t be able to identify the two problems anymore. I’ll have to upgrade it to use dwarf data one day or another.
On a few technical notes, I’ll have to rewrite the script not to sue sqlite but PostgreSQL instead; SQLite is way too slow and seems to waste too much space even handling my quite reduced set of 1006 packages, I don’t want to know what might happen to Patrick’s box if he runs it over the extracted binpkgs of his tinderbox. This will use up some of my free time during the week, but having it ran on a huge amount of packages might really help finding more cases like the ones I already documented and opened a bug for.
Considering that this is eating up the little free time I have at the moment (I can’t remember last time I had time to lay on the bed watching some good anime), I’d quite like at least some feedback from users, not only from developers 🙂 – especially considered the amount of ranting we have witnessed in the last weeks about developers not talking to users enough… I always ask for feedback but rarely get any!
Also my Sony Reader should arrive soon, so I’m considering the idea of buying something from O’Reilly to start looking at things I don’t know yet about; if somebody read those before, I’d gladly take suggestions: Linux System Programming by Robert Love (well, not an area where I know nothing, but it might be interesting to see if Love has any suggestion there that we don’t widely apply yet), Postfix: The Definitive Guide (I know nothing of postfix, so this seems a decent start, and I want to know about it as I’m using it on this server for the xine bug tracker), and CJKV Information Processing (I don’t speak any of those languages, unfortunately, but I find those problems interesting, as they reflect on a smaller scale for Italian and most of european languages too).
Everything that keeps my aging system (Athlon XP 2600) snappy is absolutely brilliant in my opinion ;)I always wondered about how much effect on startup time the symbol resolving has, are there any charts on that?Well whether people appreciate your work or not, I do and I learn a lot about C and C++ here so that’s always a plus 😉
Hi,I’m a gentoo user, that found your blog just recently. I really enjoy your posts, even though I only understand the big picture. It is informative, fun to read, and makes me happy to see talented people taking care of gentoo.I’ve been toying with prelink and stuff for a long time. Then it was said that hidden visibility is the cure, so I’m glad that things are moving.As a sidenote, I’ve once read somewhere that dyn-linking is worse for C++ (so Qt and KDE in my case mostly), maybe you could elaborate on that in a post.
Thanks for your great posts on the subject. It contributed to my understanding and will hopefully help in writing better programs.I came across visibility problems a couple of days ago when I found that stdexcept were not being recognized. For more info, see this posthttp://www.dribin.org/dave/…and this pagehttp://gcc.gnu.org/wiki/Vis…
You ask about opinion on books. From this list I have read the postfix one but only briefly. my first impression was very good and as far as i remember this book was about very many aspects of postfix configuration so you should get many useful informations from it.Greets
Again a usefull post. I really like your practical approach of things. Most of the blogs I read are really technical about what should be done etc.It kinda like the difference between people that code a small program (in c/c++). There are people that do not free stuf because they say well it will be done in the end anyway. And people that do free it. Even if it is right before the program exits. Just so they can use valgrind better and because well it is the right thing to do.I understand that feedback on posts really helps to keep up the spirit. However I do not reacat that much mainly because I let my feedreader fetch everything and read a lot on the way (in the bus/train).However there isn’t a single week that goes by without me and some of my friends at least discuussing one of your posts. So they really are greatly appricaiated.Now back to the main issue.You technical posts have nice direct effect. Which I’m sure if you played around with them enough will insert in for example xine. That would mean that I can see the direct effect in each version improvement.So keep up the good work!
I really love the kind of technical posts that you’ve been making recently. As on of the developers of Tilda (a GTK+ drop-down terminal) I really get something from the posts. For example, I tried out the trick you posted a while back about using some gcc flags and –gc-sections to the linker to help find some code that was never used in our executable. Previously, I basically did this by hand, by commenting out functions.Also, of course, I’m always trying to reduce memory footprint. Your posts about using pre-generated tables, while not quite directly applicable to me, still helped me find some areas to share some memory between processes.Thanks!” rel=”nofollow noopener” title=”http://tilda.sf.net>Tilda (a GTK+ drop-down terminal) I really get something from the posts. For example, I tried out the trick you posted a while back about using some gcc flags and –gc-sections to the linker to help find some code that was never used in our executable. Previously, I basically did this by hand, by commenting out functions.Also, of course, I’m always trying to reduce memory footprint. Your posts about using pre-generated tables, while not quite directly applicable to me, still helped me find some areas to share some memory between processes.Thanks!”>
I think you’re doing important work here. Best result will be if you get a few upstream developers to understand the issues so that the knowledge may spread. This is stuff which is very usefull but not taught in your average book or university…
I started reading this blog a couple of weeks ago, and I’m really enjoying it. You’re doing a lot to help various projects, and writing very informative posts about how you do it. I don’t understand all of it, but trying to learn is part of what makes reading this fun. Thanks for your time in writing the blog and coding xine and others.
I’m not really technically accomplished to understand the details, but i get the bigger picture…I normally read all posts on planet.gentoo.org, and i find that the developers of gentoo seem to very nice and skillfull people, keep up the good work!The few times i’ve had contact with developers they are always eager to help and they really care :)Sometimes it’s a bit scary to contact developers, i feel like i’m stealing time from them… As someone mentioned in a post recently; if developers express there opinions about being contacted in some easy to find place, that would for me make the step to contact developers shorter.Keep those nice posts coming!
I read your blog for a long period of time. And I’m really excited by what you do, how you do it and yours productivity.And moreover your posts motivates me to do similar things. Probably I will try myself as a gentoo/gnome developer/maintainer.Please keep going!And thank you for your efforts!
Linux System Programming by Love was a disappointment — a waste of money. A review here:http://drear.iki.fi/blog/bo…
Ditto tante.If, as you say, that your suggestions will makes executables smaller, faster, cleaner with minimal/no side effects then where is the down side?Could you perhaps comment on any possible downsides? Why doesn’t everyone do this alread, or is that like asking why everyone doesn’t drive safely?As a few other commenters have mentioned, your posts are understandable, clear, and point to possible improvements in Gentoo; what more could a user ask for?! (When I see the little flaming head I always stop and read)
Hear, hear. More love over at planet larry, too 😉
I’ve made a simple tool for simplify detection of packages which are likely to benefit from visibility annotations (https://github.com/yugr/ShlibVisibilityChecker). Sadly I haven’t seen much interest in upstream projects (half of patches stayed unreviewed).