I’m surprised that my previous post about rubygems got so much coverage, since I really didn’t find I added much to the common knowledge about RubyGems shortcomings. It’s not like I’m just piling up together with the Debian guys on the “gems bashing” just for fun. My switch away from gems for the packages I maintain started a few weeks ago already, and I’ve spoken with Hans about this at the time too. Now, let me try to address some of the concerns that people seem to bring up, since I’m not the kind of person who throws a rock and then hide to see the results.
As Alex said, it is actually possible to patch up Ruby Gems; and indeed I knew how to do that myself, but… it takes unpacking and repacking the files, which is generally a waste of CPU time and that is not really something that I would call “feasible” just to fix the code, when we do the same thing without any kind of issues for a huge amount of software already. Also, as he also pointed out, log4r and other gems don’t really use the tar format, instead it seems to contain some YAML-formatted list of base64-encoded files. Not nice, no.
On the other hand, Elias added that the problem is shared by all common toolkit-specific package managers: PHP/PEAR, Ruby/Gem, Python/Egg, Perl/CPAN, TeX/CTAN. This is probably right, although I think I have something to add here since lots of people seem to think that CPAN and RubyGems are on the same page, when they certainly are not, as Debian developers already pointed out. As for what concerns Python Eggs, and PHP PEAR, I have no experience working with them, while I know the Gentoo Python team does not like Eggs for just about the same reason why Ruby team does not like Gems.
As Wouter Verhelst points out:
There’s CPAN.pm, which does much the same thing as RubyGems; but much of CPAN can be easily turned into a Debian (or RPM, or whatnot) package. The few cases where it can’t, you’re usually dealing with a broken CPAN package, anyway.
He’s perfectly right here, CPAN modules can be easily turned into distribution-specific packages, included ebuilds thanks to g-cpan. The reason for this is that instead of inventing its own freaking file format, CPAN uses standard tarballs or ZIP packages, and a specific structure of the files inside of the package. Once you know how the structure for a package is, implementing downstream-specific packaging of such tools is very quick, and still allows for all the versatility that downstream package manager require.
For the little I know about it, I think CTAN does just the same, and indeed Wouter states that too:
There are similar things for Python (egg), and, heck, even TeX (CTAN). The fact that other scripting languages have proper and working packaging systems only outlines that there really isn’t an excuse for the horrors of RubyGems.
I think that Wrobel’s post is indeed a very important reading because it summarise the fact that the argument that Gentoo and Debian have against RubyGems is not that there shouldn’t be a language-specific package manager for Ruby, but rather that we would have certainly liked a better-designed package format that could suit both the system-independent packaging and the downstream packaging at once, allowing for proper integration between modules installed by users and by the distribution.
The fact is that beside gems there is no vast and common standardisation of practises between Ruby packages, which makes the role of distributions much more complicated than it has been historically for Perl, Python, TeX and the like. What do I mean? Well, take extconf.rb and setup.rb files that are often used to build binary C-based extensions for Ruby: they are not generated by tools like autotools or distutils, and instead they are copied and morphed from package to package, causing them to have subtly different syntax and stuff like that. To use a DESTDIR-like parameter (to install in a subtree before packaging) some require you to specify that during install phase only, others will require you to pass it on all the parameters. One would expect that Rake could have brought some more standardisation in packaging, but it’s not the case either.
For once, Rake does not usually have an “install” task, and even less one that taking a DESTDIR parameter to install in an offset. This is something I’ll probably try to find a solution for once Ruby-Elf arrives to a point where a release is needed, because I don’t intend to use RubyGems as my primary form of distribution, quite obviously. But it does not limit to that: even the tasks that are usually common between packages don’t follow the same interface. Some packages will use “rake test” for running the testsuite, others “rake spec” and others “rake rspec” because they don’t use Test::Unit but rather Spec. Some packages will build the HTML documentation with “rake doc” and others with “rake rdoc”.
I think that the underlying problem here is that Ruby has been really the most “magic” of the languages. While this is very nice for the developers, it is not always good; sometimes having too much magic around can make integration or debugging very hard, just like automagic dependencies make it difficult to properly package software, Ruby magic can be a hinderance when doing something that goes a bit away from what the mainstream developers do.
Now, since just criticising software is rarely useful, let me try to explain what is my plan to resolve, or work around, the issue. First of all, I’ll see to work with Hans and Alex, as soon as I have time, to find a way to simulate the presence of a gem when the package was installed by the ebuild using a tarball, so that there is no loss of magic when using the ebuilds themselves. Then I’m going to try preparing a distribution-friendly “install” task for Rake, as I said first and foremost for RubyElf but I’m going to release it as-is so that other projects can use it too, and try to standardise it.
But these are little steps; what I’d like to do, but I lack the time to, is writing a specific about packages installation and design for Ruby so that a project following that specific can not only build gems for “magic” installation on heterogeneous systems, but also prepare tarballs that distribution can turn into proper packages without having to deal with black magic. This would not mandate a particular file format like gems are doing, but rather a particular directory structure or Rake-based interface. Once the same exact steps can be run for any given package following the specification, it’s going to be just enough for the distributions.