Why Ruby ebuilds are not autogenerated

From time to time somebody drops in on #gentoo-ruby, or mails the Gentoo Ruby team alias, either asking about, proposing to help with or outright attacking us for not having a tool that automatically generates ebuild files for gems. Call it or design it like you prefer, but in general, the whole question boils down to “why do we think ebuilds should not be autogenerated?“.

At a first glance, the Gem specification (gemspec) provides information the same way an ebuild does, for the most part, so why shouldn’t it be feasible to produce an ebuild out of that? Well there are many reasons, but the main one is that gemspecs don’t provide all the information we need, followed straight by gems don’t mandate enough of a standardised interface.

These two problems find their roots in the RubyGems approach of creating a format and managing it as a single piece of code. As I have written many times, the RubyGems project should have split their focus and handled them independently rather than as a single, monolithic black box:

• discovery of the installed extensions is something RubyGems handle quite well, most of the time at least; as a standard for install location, and requests of particular libraries, it is a method good enough that in Gentoo we have no intention whatsoever to replace it; indeed the RubyNG eclasses install in the same location, and provide a compatible registration of the package; the problem here is the fact that the registration is not independent from the way the code is installed, but is a gemspec by itself;

• the package distribution format used by gems is problematic as we said before; some older gems (luckily I haven’t seen one in a long time) used to provide base64-encoded archives rather than the double-tarball they come in right now, and even the double-tarball is a bit nasty to deal with; on the other hand I can see why it was designed that way and it’s not all bad;

• what I think really suck is the gem package manager which is the issue that both Debian and Gentoo have with the whole system: it does not comply with the usual standard for distributions, and it shows that it was designed for platforms where no package manager is available (namely Windows and OS X).

Because these three objectives haven’t been separated, only the strict internal requirements between the three of them have been taken into account. If the package distribution format was instead designed by consulting with the developers of distributions’ package managers, it would most likely have taken a quite different direction, by providing the metadata we currently lack, to begin with.

For instance, the specifications don’t tell us which license the gem is released under, they don’t tell us which dependencies are mandatory and which are optional, they often don’t tell us the correct test dependencies either.

But most importantly, the whole RubyGems infrastructure mandates no interface. How do you run tests? Most of the times, you can just run “rake test” and it works, good, but is it always the case? Of course not. Even if a small set of developers do create the aliases when they use rspec or other software for testing, there is no mandatory interface for testing, and that is quite a problem.

Again, as I said above, if the three goals were to be split properly, then it would have been quite easy to leverage the abilities of the various package managers into the gemspec format: all distributions want to know the license of a package, and most will want to have a mandatory interface for build, test and install. If you had the gems as package-manager-agnostic packaging for Ruby extension, you’d have win: distributions would use autogenerated packages for their own repositories, and the RubyGems manager would add integrated external packages for the stuff that was missing.

Unfortunately, life is nowhere near as easy. But since I’m gathering more knowledge about the common tasks that Ruby extensions have right now, I’ll probably resume my “Ruby Packaging Specification” project and try to give pointers on how to achieve a more standardised interface for gems so that, slowly maybe, we can reach the goal of having gems play nice with our package managers.