This Time Self-Hosted
dark mode light mode Search

Why Ruby ebuilds are not autogenerated

From time to time somebody drops in on #gentoo-ruby, or mails the Gentoo Ruby team alias, either asking about, proposing to help with or outright attacking us for not having a tool that automatically generates ebuild files for gems. Call it or design it like you prefer, but in general, the whole question boils down to “why do we think ebuilds should not be autogenerated?“.

At a first glance, the Gem specification (gemspec) provides information the same way an ebuild does, for the most part, so why shouldn’t it be feasible to produce an ebuild out of that? Well there are many reasons, but the main one is that gemspecs don’t provide all the information we need, followed straight by gems don’t mandate enough of a standardised interface.

These two problems find their roots in the RubyGems approach of creating a format and managing it as a single piece of code. As I have written many times, the RubyGems project should have split their focus and handled them independently rather than as a single, monolithic black box:

  • discovery of the installed extensions is something RubyGems handle quite well, most of the time at least; as a standard for install location, and requests of particular libraries, it is a method good enough that in Gentoo we have no intention whatsoever to replace it; indeed the RubyNG eclasses install in the same location, and provide a compatible registration of the package; the problem here is the fact that the registration is not independent from the way the code is installed, but is a gemspec by itself;
  • the package distribution format used by gems is problematic as we said before; some older gems (luckily I haven’t seen one in a long time) used to provide base64-encoded archives rather than the double-tarball they come in right now, and even the double-tarball is a bit nasty to deal with; on the other hand I can see why it was designed that way and it’s not all bad;

  • what I think really suck is the gem package manager which is the issue that both Debian and Gentoo have with the whole system: it does not comply with the usual standard for distributions, and it shows that it was designed for platforms where no package manager is available (namely Windows and OS X).

Because these three objectives haven’t been separated, only the strict internal requirements between the three of them have been taken into account. If the package distribution format was instead designed by consulting with the developers of distributions’ package managers, it would most likely have taken a quite different direction, by providing the metadata we currently lack, to begin with.

For instance, the specifications don’t tell us which license the gem is released under, they don’t tell us which dependencies are mandatory and which are optional, they often don’t tell us the correct test dependencies either.

But most importantly, the whole RubyGems infrastructure mandates no interface. How do you run tests? Most of the times, you can just run “rake test” and it works, good, but is it always the case? Of course not. Even if a small set of developers do create the aliases when they use rspec or other software for testing, there is no mandatory interface for testing, and that is quite a problem.

Again, as I said above, if the three goals were to be split properly, then it would have been quite easy to leverage the abilities of the various package managers into the gemspec format: all distributions want to know the license of a package, and most will want to have a mandatory interface for build, test and install. If you had the gems as package-manager-agnostic packaging for Ruby extension, you’d have win: distributions would use autogenerated packages for their own repositories, and the RubyGems manager would add integrated external packages for the stuff that was missing.

Unfortunately, life is nowhere near as easy. But since I’m gathering more knowledge about the common tasks that Ruby extensions have right now, I’ll probably resume my “Ruby Packaging Specification” project and try to give pointers on how to achieve a more standardised interface for gems so that, slowly maybe, we can reach the goal of having gems play nice with our package managers.

Comments 3
  1. I just want you to know that I think what you’re doing here is great! Thank you 🙂

  2. We have a similar problem in the Haskell world: going from your description, the Cabal format for Haskell seems to be nicer than Gem’s and we _do_ have a tool to generate ebuilds…… but we still have to tweak them to make sure they’re right. This is to ensure dependency ranges are valid (since ranged dependencies are encouraged for Cabal specifications but are not available in Gentoo), non-Haskell dependencies are included (whilst Cabal does allow you to specify required C libraries, how would an automatic ebuild generator convert a C library called icuuc into dev-libs/icu ?), matching up build-time flags to USE flags and also to install extra documentation, etc.In a sense, the Cabal format is at times also under-specified in terms of an interface as there’s no official requirement on what to call the testing flag, what to do with READMEs, etc. But we’re lucky that Cabal itself is a library rather than a tool (there is cabal-install as well, but we don’t use it for the ebuilds) and that the lead Cabal hacker (Duncan Coutts) is an ex-Gentoo developer.

  3. I’d welcome a tool to make ruby ebuilds. autogen…attempt to fix if busted , or at least get to 95% of the way their as a RAD prototype tool….g-ctan or g-cpan has been a welcome alike tool ..even still as a tool it may be of use to the devs in geting faster to packaging of new ruby gems. for a Tinkering of those with a solid idea of whats going on, but not the tool most gentoo users will abuse.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.