Important! Do not update to Python 2.6.5_p20100801

Seems like someone pulled another breakage, almost an year after the past one. Please do not upgrade to this version of Python, and if you have problems like the following:

>>> Emerging (1 of 1) dev-lang/python-2.6.5-r3
Traceback (most recent call last):
  File "/usr/bin/emerge", line 42, in 
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1555, in emerge_main
    myopts, myaction, myfiles, spinner)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 434, in action_build
    retval = mergetask.merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 914, in merge
    rval = self._merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1222, in _merge
    self._main_loop()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1369, in _main_loop
    self._poll_loop()
  File "/usr/lib64/portage/pym/_emerge/PollScheduler.py", line 134, in
_poll_loop
    handler(f, event)
  File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 151, in
_output_handler
    buf.fromfile(files.process, self._bufsize)
IOError: [Errno 11] Resource temporarily unavailable

Then please see bug #330937 — ~~at this point in time I have no idea how to solve if you updated already~~, it’s 3:50am and I’m actually here just because I made a mistake rebooting Yamato and found this out.

Update: we have a quick hotfix for you to apply if you reach this point:

wget http://dev.gentoo.org/~ferringb/fix-python-2.6.5_p20100801.patch -O - | patch /usr/lib/portage/pym/_emerge/SpawnProcess.py

it’s a one-liner, just execute it, then you can simply re-merge Python 2.6.5-r3 and Portage to have the pristine system.

Gentoo Failed Us Again

Kudos to Markos who basically gave me the title for this blog post!

I’ve spent the past week or so away from computers, I’m having some personal trouble, tied with bad migraines that would have burnt the hell out of me. I came back to updating my systems today, and I received a nasty surprise. Unmasked libpng 1.4 is wrecking havoc on so many systems that it’s not even funny.

I’m not complaining about the fact that we’ve finally unmasked the new libpng, it was needed and we should probably proceed on getting it stable soonish as well. What I complain about is that we’re hitting the same obstacles we hit with libexpat:

we still don’t have enabled --as-needed by default, which would reduce considerably the amount of packages that actually need to be rebuilt after such an update (and, by the way, not using --as-needed also increases tremendously the chance that some program will be loading both 1.2 and 1.4 versions, with the usual trouble of symbols’ collisions);
we still haven’t solved the problem with libtool archives , requiring the rebuild (or nasty hack) of a number of packages for no good reasons.

The worst part is that I have been preaching about both things for a while, a few years I’d say, and yet we have not gotten our heads out of the sand, so we hit users in the face after this kind of updates. Still.

Using --as-needed, only a fraction of the packages installed in the system will actually link against the libtool file, and only those would need to be rebuilt; without it, it’s very likely almost all the libtool-using packages, as well as most pkg-config using packages will be linking in libpng as a dependency of other libraries, such as GTK+ or Qt. And since you will start updating from those libraries, the newly-started packages will have problems because both libpng versions will be loaded at the same time: once from the library and once from the application.

For what concerns the .la files, the problem is mostly at buildtime and it makes it very very difficult to get out of the mess caused by the update, as a number of packages will start lamenting of missing targets for -lpng12. The solution for this would be to, obviously, carefully remove .la files within the ebuilds; this way we reduce the chances that the dependencies end up polluting packages that would, otherwise, have no involvement whatsoever with libpng.

Unfortunately, removing all the libpng file indiscriminately is a Bad Idea™ (and yes, I know some people experimented with that, I still maintain it’s a bad idea!). What you want to do is to reduce their impact as much as possible, but to do so you have to do some extra work, and that requires developers to understand the problem and accept working on a solution, even a temporary, imperfect one, to avoid staying in the problem area.

Do you remember when I checked eog (Eyes of Gnome) .la files resulting in stating clearly that they are totally useless? Well, eog still installs them; sure enough they are not excessively important in this situation, as they are not linked against, but they will create false positives in revdep-rebuild for instance. Even my flowchart has gone mostly unused by developers.

And we still don’t have any way to sanitise those files within Portage; lafilefixer does solve some stuff, but it’s not part of Portage proper, nor it’s integrated with it. If you want, in the future, to reduce your system’s pollution, do something like this:

# /etc/portage/bashrc
post_src_install() {
    lafilefixer "${D}"
}

This way the files will be sanitised before being merged in the system, and you won’t have to fix them manually.

Will we ever learn?

Brace for impact

Today I was looking around for a bug in autoconf, and I noticed one interesting bit out of the NEWS file of the current git version:

Present But Cannot Be Compiled: Autoconf will now proceed with the compiler’s result if a header is present but cannot be compiled. The warning is still printed, and you should really fix it by providing a fourth parameter to AC_CHECK_HEADER/AC_CHECK_HEADERS.

This is a tremendously useful thing to know before autoconf 2.64 is released, which is hopefully not too soon. The reason for this is that finally, after years of having that as a warning, to the point that some projects even ignored it altogether, the new autoconf will start ignoring header files that cannot be compiled, for whatever reason. This is useful since it ensures that headers are not detected that lacks proper dependencies. Unfortunately this also means that any software that currently relies on header files found without compilation will change behaviour. In particular, warnings like the following need to be addressed before the packages get to use autoconf-2.64:

checking bluetooth/bluetooth.h usability... no
checking bluetooth/bluetooth.h presence... yes
configure: WARNING: bluetooth/bluetooth.h: present but cannot be compiled
configure: WARNING: bluetooth/bluetooth.h:     check for missing prerequisite headers?
configure: WARNING: bluetooth/bluetooth.h: see the Autoconf documentation
configure: WARNING: bluetooth/bluetooth.h:     section "Present But Cannot Be Compiled"
configure: WARNING: bluetooth/bluetooth.h: proceeding with the preprocessor's result
configure: WARNING: bluetooth/bluetooth.h: in the future, the compiler will take precedence
checking for bluetooth/bluetooth.h... yes

Just so you know, this code comes from kdepim-3.5 build, which means that the old KDE build system is once again screwed (it’s very useful to change build system when you can’t even use one properly, the way KDE’s CMake-based build system fails at finding Ruby shows that even without autotools, upstream can make a huge mess, like depending on akonadi to be able to install Umbrello…).

The list of packages having these warnings available is actually not too long which means it should take little time to fix them, the problem is that I think I remember that previously it had a slightly different output message, which means my grep might not have hit properly, I’ll have to investigate that more deeply. Also, these are only the problems identified in Gentoo Linux, I know for sure that there are many more in Gentoo/FreeBSD, since the FreeBSD code does not express all the implicit dependencies between the headers (which is something that I sincerely can’t understand from time to time).

Unfortunately, there is no straight and final recipe to fix these problems, since they can easily be caused b y various entirely different options, for instance it can be a header that is not included when it should be (the most common case, which is what the autoconf news file reported), but it can also be a C99 header included using a compiler set to C89, like in the case above, indeed if you check the config.log file for the above build you’ll see this:

configure:34904: checking bluetooth/bluetooth.h usability
configure:34921: i686-pc-linux-gnu-gcc -c -std=iso9899:1990 -W -Wall -Wchar-subscripts -Wshadow -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -DNDEBUG -O2  -O2 -pipe -Wformat-security -Wmissing-format-attribute  -DQT_THREAD_SUPPORT  -D_REENTRANT conftest.c >&5
In file included from conftest.c:97:
/usr/include/bluetooth/bluetooth.h:117: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
/usr/include/bluetooth/bluetooth.h:121: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'

And the two lines contain these two definitions:

/usr/include/bluetooth/bluetooth.h:117:static inline int bacmp(const bdaddr_t *ba1, const bdaddr_t *ba2)
/usr/include/bluetooth/bluetooth.h:121:static inline void bacpy(bdaddr_t *dst, const bdaddr_t *src)

The inline keyword is not available in the standard requested by KDE’s build system for the C language, and thus using that header is not correct. But for the sake of finding a solution, it’s very well possible that most KDE packages checking for bluetooth.h could be made not check for it at all (the way KDE build system checks for stuff for single modules in the main configure file is probably one of the most obnoxious mistakes in that abomination).

Now I guess it’s time to start preparing bugs so that we are not unprepared for when this will actually be enacted!