Okay I’d sincerely would like to stop blogging about Ruby and actually working on making it better, but it seems like that one particular user is trying to start up fires all around the Ruby Team to push for his agenda… without seeming to care at all about the results. So here comes a list of common questions and mostly-official answers.
I forgot one of the most important questions! Added afterwards!
Why is Ruby 1.9 masked? It’s stable upstream!!!! The exclamation marks are there because that’s how we receive them oftentimes. Sure, Ruby upstream seems to declare that Ruby 1.9 branch is “stable”, by their standards already, which I’ll have to tell you all, are not very solid. To be precise, they mark 1.9.1 stable, while 1.9.2 is still “development” and is currently a Release Candidate. While the code for 1.9 sounds more solid, the changes between 1.9.1 and 1.9.2 are not trivial, and wouldn’t have been considered for most other projects worth of “minor versions”. To make a comparison, they really look like the difference that there would be between GCC 4.5 and GCC 4.6, while the changes between 1.8.7 and 1.9.1 are like those between GCC 4 and GCC 5.
What we’re actually targeting for unmasking at some point in the hopefully not-so-distant future, is 1.9.2, not 1.9.1, so no, our target is not considered stable upstream yet.
Just unmask 1.9.1 then! Unfortunately, that’s not feasible. The reasons are long and are (partly) on our side: we’ve not had a proper way to handle extensions’ packaging until I created Ruby-NG in May 2009 – all the previous ebuilds used various series of hacks and dirty tricks to allow side-by-side installation of Ruby packages, but none actually worked properly. Even worse for the gem-based packages. From May 2009, we’ve been refining techniques, eclasses and porting packages, but the work is still not complete. There will be much more to deal with before unmasking. And that means that our most likely candidate is going to be 1.9.2 rather than 1.9.1.
This does not mean that we’ve not tried targeting 1.9.1; I actually did a lot of porting to 1.9.1… and now that we’ve turned to look at 1.9.2 I had to do further porting.. wasting twice the time to identify and fix issues. Trust me it’s not funny.
Why doing the work? Leave it to Rubygems! Haha, very funny. I’ve written a long time ago about the reasons why we don’t consider the Rubygems package manager apt to production use. Even though a number of people find it just perfect, we’ve got our reserves, and feel like Gentoo should aim to provide higher-quality packaging to our users, since we actually can.
In particular you can manage a remote system with binary packages without having to keep around the development tools thanks to our packaging, as you’d be building the binary gems together with normal Gentoo packages, and abiding to the settings you provide to Portage.
Of course you can use the standard gem command, but if you do so, particularly as root, you “void the warranty” and the Gentoo Ruby team won’t respond for any problem you might experience.
Why do I see so many tests failing? I didn’t see them at all before! You broke something! Not really; by default gems don’t execute tests; sure there is an option in the Rubygems package manager that provides support for running tests, but the number of gems that actually specify their test dependencies, and the way to run the tests, is risible. And with our previous packaging system, the tests simply weren’t executed. So all the failures you might experience with the new ebuilds based on Ruby-NG, are for the vast part coming directly from upstream.
To be fair to most upstreams, some tests are either designed to work on older Ruby versions, on other operating systems (such as Mac OS X that is case-insensitive — bundler had a problem related to that!), or with a particular series of installed gems. Which means that they might not have seen the failure at all. And some failures are actually coming from our environment; in particular lately the tinderbox is reporting me a spree of failures happening when test-unit 2.1.1 is installed for Ruby 1.8 (and would for Ruby EE if I was testing that).
We’re doing our best to actually fix the test failures, sending them upstream, as you can see from my own GitHub page that has forks for a number of Ruby projects for which I sent one or two patches to fix tests. This, though, is often a non-trivial yet boring work and we’re just four guys trying hard to get results.
Why is package #{foo} not available for 1.9? Upstream says it works! This usually has Rails (2.3) in place of #{foo}… yes upstream sometimes says their package works with 1.9, and most times they are right, but some other times, they are definitely not. Rails is a good example of the latter. It “works” in the sense that you can use it for some of its features, it does not work as in “everything is fine and smooth”. Even its own tests, for what concerns Rails, are totally bogus on 1.9. It gets worse for binary extensions (bindings).
There is a forked gem/patch to get #{bar} binary extension to build with 1.9, why you insist it’s not working on 1.9? This gets fun, and I actually written explicitly about it — sometimes the patched versions do exactly what it’s written on the box: they build on 1.9, but that says nothing about them working at all. Since a lot of symbols (functions and macros) have gone from 1.9, even fixing the extension to build might still entail using symbols that are no longer available, through implicit declarations. The original Ruby 1.8 and 1.9 build systems leave that be, and depending on the settings of the system this will lead to either non-loadable extensions, or runtime suicide of the Ruby process, because of missing symbols.
Personally, I don’t like having timebombs in my packages, so under the assumption that I’d rather have a build-time failure – or no package at all – than one that can kill my software stack at runtime, I’ve made our Ruby packages inject -Wl,--no-undefined so that similar problems are caught at build-time already. At least two packages that patched for Ruby 1.9 support in the overlay before failed this very test and were thrown out of the Ruby 1.9 compatibility.
Why can’t I merge the packages directly if I have FEATURES=test (or USE=doc) enabled? Since we’re actually running tests, we’ve got to depend on the packages that are used by the tests themselves; to be on the safe side, when there are optional tests we try to push all of them to be run during the test phase. Unfortunately because of the nature of the gems themselves this way too often lead to circular dependencies; this is why I call it a dependency web rather than a dependency tree, nowadays.
When merging Ruby 1.9, it’s telling me that Rake and Rubygems are being dropped, why is that? Ruby 1.9 comes with a bundled copy of Rake and Rubygems, but those are quite older, and a number of packages required much newer versions of either, or both. We could allow for an override for those, but it would be much harder than simply removing them and adding the dependencies over the normal Rake and Rubygems that install for 1.9 as well. Actually, the same destiny is there for JRuby that bundles even more packages.
Portage is reporting that the package has been built without respecting LDFLAGS, why’s that? Well, while I’m the first person who’d want packages to always respect flags, Ruby packages are very tricky. Sometimes we can rewrite CFLAGS/LDFLAGS directly on the emake command line, but there are many cases where that is not possible, so the actual LDFLAGS used for rebuilding Ruby itself are used. This is one of the things that we could use some help, or some more dedicated time.
What about JRuby? You started so well on it and is now slowing down, why? Mostly because a number of issues upstream required newer JRuby versions altogether; then they fractured support in an even higher number of further packages with mixed Java and Ruby code. I’m no Java expert to begin with, and keeping up with the development is not something I have very much time to tackle. The other members of the team seems highly uninterested in JRuby so far, thus you get what there is time for.
The package #{fnord} is installing itself as a gem, but it’s using a GitHub tarball rather than the gem file, why? Once again it has to do with the fact that gems are not required to provide or execute tests; while a few gems feel safe enough to let users run their tests, there are many that simply do not provide tests with their gem files. To solve this, we go around it, and download the code directly from GitHub, and use that for the packaging. Thankfully, GitHub got their shit together and is now feasible for us to keep using their download service. That is, if the upstream developers actually bother tagging their releases. Sigh!
How can I help to get Ruby 1.9 support in? Again there is a dedicated blog post from me that can be summarized as “we need more testing of the packages, reporting bugs upstream where it make sense, and where it makes sense more tests .”
There are a number of issues that need to be tackled, you can find most on Bugzilla — simply open the page and look through the bugs. Make sure that the test failures get fixed upstream, and not simply blacklisted, if possible.
I need Ruby 1.9 available right now as I’m using it in production! Tough luck, because “right now” is definitely not going to happen. Soon, maybe; right now, no. As I said there are a number of things to kink out; on the other hand, you can either help as stated before, or find someone who can be paid to help. I’m up for hire — and I’m repeating this because I’m growing tired of the few people who actually make a living out of this expecting the four of us to abide to their agenda.
I’m using Funtoo and… Stop there! The Gentoo Ruby team does not and will never support Funtoo users. Daniel decided to go on with Ruby 1.9 at a point where neither the upstream projects nor the eclasses were designed to. Whatever he did after that, we have no intention to sort out.
More questions?
