Same-ABI and any-ABI dependencies

Two really-desired features of Portage, that are important for, respectively, desktop and embedded use cases, are multilib and cross-compilation. Both of these, to be properly implemented, require Portage to discern between same-ABI and any-ABI dependencies.

This concept has been called in the past Linked-in and Executed dependencies, but I don’t like that name at all as it makes sense only for those who actually know what the two concept expects. Actually, I don’t like this name either because it confuses the term ABI as the calling convention of an architecture, and the term ABI as the compiled interface of a software library. If anybody can make sure we find a simple term for the calling convention type of ABI, it would be quite nice in my opionion.

Another good reason to get rid of the terms Linked-in and Executed dependencies is that abstracting well enough the concept, one can easily see the same code and mechanisms to be used to describe the dependencies of extension modules like Python’s and Ruby’s, that depend on the version of the interpreter they are built and ran against.

With good enough support, this would allow to handle dependencies for multiple Python and Ruby versions at once without needing strange and silly hacks like the ones present in the ruby eclasses. And would have solved the problem of PHP extension versions that caused to split them in dev-php4 and dev-php5.

Why is this distinction needed for both multilib and crosscompile? Well, let’s start with the multilib case, and see how it works.

Let’s say you want to install mozilla-firefox-bin, which is 32-bit. It will require a series of libraries built 32-bit like GTK+ and similar, but it will also require the launcher script. That launcher script does not need to be built with 32-bit ABI, as it does not interface directly with Firefox. The libraries will be a same-ABI dependencies, while the script will be an any-ABI dependency.

Even better, the script, being a script, could be declared to not have an absolute ABI, something alike RPM’s “noarch”. This would make it much easier to handle dependencies on data packages (icons, documentation, scripts).

But any-ABI dependencies can easily apply to non-scripts, that is binary packages. Let’s take for instance CUPS and the software that Canon makes available for Pixma printers. The software is only available for 32-bit systems, as it relies on some binary files that Canon does not give the sources of. On the other hand, it does not interface with CUPS through library calls, it’s instead a filter that gets executed to convert the PostScript data coming from CUPS to a format that the printer will understand. The Pixma software would then have an any-ABI dependency on CUPS, and a same-ABI dependency on the libraries it links to, like GTK+.

On cross-compiled environments, this merged with the ROOT variable support would make it much easier to build a system’s root with the minimal amount of software needed, and would solve the problem related to Perl’s cross-compiling.

A few packages need autoconf to be present in the system, but none of them need anything more than being able to execut them. When cross-compiling, there is no need to have a copy of autoconf built for the ABI of the system you’re building, you just need a copy of it that you can execute. This makes it an any-ABI dependency. Not asking a CHOST-built autoconf, Portage won’t be asking for a CHOST-built Perl either, and that would save you the headache, because Perl cannot be cross-compiled.

This of course will require a lot of changes, especially EAPI related, and finding a new, decently good syntax, for ABI-specific dependencies. In particular, there are a few issues that needs to be addressed before one can even think of implementing this:

• there has to be a way for profiles to tell Portage the ABIs a system can handle; an amd64 multilib profile will advertise support for x86_64 and x86 ABIs for instance, an x86 Gentoo/FreeBSD profile would advertise support for x86/FreeBSD and x86/Linux ABIs (through the Linux compatibility layer);
• there has to be a way to let packages inject further ABI specifications, like I said above, Ruby and Python are two perfect examples; a system that has Ruby 1.8 and Ruby 1.9 installed, will advertise support for both ABIs, while a system with just Python 2.5 will advertise that single ABI;
• cross-compiling still would require some work, maybe a replacement for crossdev could use a prefix like cross- or something like that, it would need someway to express cross-compiling tools and libraries versus cross-compiled tools and libraries;
• by default, the packages should be available on any native ABI, maybe a meta-ABI “native” would allow to say “whichever ABI the profile accept”; it could somehow cross-over with the meaning of KEYWORDS, especially for cross-compiled environments;
• different ABIs will have different paths for installation: on amd64 multilib systems, the 32-bit libraries are currently installed by the 64-bit sides on /lib32 directories, while the binaries are mixed in /bin; on a Gentoo/FreeBSD system, you most likely want to install everything prefixed in a /linux directory, while for cross-compiling support, you’d install in the SYSROOT (/usr/$CHOST prefix); as you can guess, here it would be very nice to import at least part of the extensions the Prefix project have to allow handling rbitrary prefixes; it might not need further enhancements to support Solaris and the rest (but I could write something about that), but at least it would work to have support for the arbitrary prefix in tree.

There are probably a tons other problems, and I have no clear idea on how this should be implemented as it is, but at least this could be a starting point for some discussion in this sense, and maybe Zack or Marius might catch on this and start prototyping some interaction.