Ruby woes; Rails woes; Why do we insist?

I’m sorry if my reader will have to endure another boring rant regarding Ruby and Rails in particular. Unfortunately it’s that time of the year again, when the Rails project releases a new micro version which seems more like a new minor version, instead.

The important part in the notes for this release is the fact that they decided to update some of the dependencies and bundled libraries; for instance they now use Rack 1.1.0 rather than 1.0.1 (both are in Portage, by the way). For some of these projects, there are little to no problem; updating tzinfo is something very trivial — even though there is no need for that in Gentoo, we use the system copy which is always updated. On the other hand, there is one huge change there: i18n!

The i18n gem bundled with Rails 2.3.5 was from minor series 0.1; the latest stable series is 0.3 and the development has moved on to 0.4. The problem is that all these minor versions are not compatible one with the other; which is why we have slotted 0.1 and 0.3, and have had both in tree since the conversion to Ruby-NG, with related unbundling.

With Alex working on Ruby-Enterprise the task of updating Rails went to Hans and me; since we’re both using it in production we’ve good reason to triple-check it. We partially split the sub-tasks: Hans was looking at bumping Rails proper, while I took care of a couple of side ebuilds.

First problem first, i18n itself: we’ve had version 0.3.3 in tree, but with no support for JRuby (it failed tests), which is what Rails bundles right now, on the other hand, the latest version was 0.3.7, but the gem didn’t bring in tests at all. The original idea was to tackle 0.3.3 so that JRuby would work with it, but this brought us more subtle problems: i18n optionally loads ActiveRecord, and if it finds it it opens a connection through the sqlite3 adapter – but JRuby has no sqlite3! – and then it finally failed with JRuby.

Option 2; update to 0.3.7, since I had to patch it anyway. As usual, since the gem has no tests anymore, fetch the data from GitHub, and use that, then get a local clone to do the changes. Working around the sqlite3 problem was easy: simply require the extension before ActiveRecord to make sure that it will avoid the whole ActiveRecord slab of tests on JRuby. What turned out being more complex was, instead, getting this to work as intended with Ruby EE: as my user it worked fine but it failed with Portage, with a message that definitely didn’t come out of i18n.

I ended up tracking down the problem within RubyParser; but in something quite strange: C code in a .rb file… RubyInline! The idea behind RubyInline is that you can add C code straight within Ruby, produce a C source file, and compile it to an extension at runtime, that can be then loaded into the running Ruby process. This approach has quite a few problems though, and I was hitting one in particular already: the generated extensions – saved in ~/.ruby_inline – are discerned by signature of the methods, and by name of the class, but not by Ruby implementation being used.

The bottom line here is that the extension generated with Ruby 1.8 were being loaded by Ruby Enterprise (as they resulted up-to-date)… but the two implementations are not binary compatible, at all. Which caused the problem. I’ve patched RubyInline to take into consideration the RUBY_DESCRIPTION variable when choosing the module’s name, and updated the tests. This way, it can safely work with both Ruby 1.8 (MRI) and Ruby Enterprise. Let’s not even go with Ruby 1.9 as it doesn’t work there at all.

To avoid having further problems with this, I took into consideration what the documentation stated: that it is possible to pre-build the extensions into the packaging. The documentation states that hoe should take care of it all by using rake package INLINE=1 but… that doesn’t work. Hoe 2.6.0 – which is developed by the same guys of RubyParser and RubyInline, by the way – has changed the rules a bit, so you have at least to load the inline plugin; unfortunately even if you do so, it doesn’t work at all: the plugin is hosed, so I hacked a bit at it to get it to work.

Once I was able to get it to work, though, I understood how it worked: it simply let the code load, so that RubyInline could do its magic, then copied the files from ~/.ruby_inline into the packaging… silly as that is, I changed the ebuild to replicate the same. Unfortunately, given my patch makes the extensions dependent on the RUBY_DESCRIPTION constant, it means that you have to rebuild ruby-parser every time you update either Ruby 1.8 (MRI) or Ruby Enterprise. But at least then you won’t end up hitting the C compiler at runtime.

In parallel, Hans found one very nasty thing with the new Rails: the new i18n has a completely different interface… and that interface does not work properly with Ruby-Gettext, so if your Rails application is using the latter, the update will break you badly. Have fun with that.

Sigh, but it’s not all done here! By looking at the debug log of the i18n testsuite (for double-checking RubyInline working state) I found that ActiveSupport failed to load the JSON C-based extension, and fell back to the pure Ruby (“json_pure”) extension, even though I was sure we were building the C extensions. A quick check there turned up the problem: we were installing the shared objects in the wrong path, it is fixed now.

And speaking about JSON… the new Rails adds support for an alternative to the JSON extensions: yajl-ruby, the bindings for the (otherwise C) yajl package. I knew the package already because it is dependency of a new proposed package for which I received a bug some weeks ago, and it’s also used, in alternative to JSON itself, by the Storable gem — which in turn is part of the Rudy dependency tree. For these reasons, I already asked Tomáš to review the yajl package from Sunrise and polish it up, building up the yajl-ruby ebuild was definitely easy thanks to that, although I had to patch it up so that it wouldn’t require ActiveSupport for the tests, because of one, single, stupid function.

Speaking about yajl, though, I’m not sure if I’ll be suggesting it to anybody: cowstats suggests it has a 64KiB .bss section, whereas JSON is way below 4KiB (the page size)… it might not say all the story about memory usage but it certainly looks strange to my eyes…

Sob, I’m not sure why I still work with Ruby, with all this mess that is there…