JRuby in Gentoo, a bumpy road

So as you might remember one of the reasons for my restructuring of Ruby eclasses has been the chance of getting “proper” support for JRuby in Gentoo. With proper I mean that the packages are declared to support specifically JRuby, and at the same time get tested so that they actually work with it. For this to work out properly it also requires that the testsuites for the packages were written in such a way to work fine when run through JRuby. But I already ranted about that so I won’t go into those details again now.

Instead of writing about the Ruby-NG side of the fence, I’d like now to talk about the (to me, much more unknown) Java side of the fence. Now, I know near to nothing about Java packaging. I know my way around the Java language, sort of, but not much about the packaging side of it: ant is mostly black magic, and the whole mess with Jar files, manifests, and dependencies are huge question marks in my view.

So the first problem, is that while JRuby has been in tree for a while, there was near to no way to make sure that it worked as intended, before. This resulted in pretty erratic behaviour with it (like the FFI-based libraries not working at all); with the root causes hidden by sub-projects as well (in this case, jffi was being installed without the needed JNI library). This is also often tied with the way JRuby is developed: it’s modularised, but the projects are sometimes only used by JRuby itself, so without it working, it’s almost impossible to tell if the rest is working as intended either.

There is also the problem of packaging what was not available before; for instance I found out that JRuby now needs a Java implementation of a YAML parser, yecht written by a JRUby developer Ola Bini, which is bundled by JRuby itself, but we should have been packaging ourselves. Thanks to Fabiano “elbryan”, I got an ebuild for yecht itself… the problem is that the version shipped with JRuby also contains the classes needed for the JRuby bindings, which aren’t available when building the Jar standalone. D’uh!

Other problems are really tied to the projects standalone: as I said above, we had some trouble with JFFI and the fact that we didn’t install the JNI library (an ELF shared object) that it needs to work. Now we do, but… every time we start jruby, whether we’re going to use FFI or not, we’ve got to call into java-config (which is a bash call into a Python script), to ask it where the jffi library is so that we can set a property… it doesn’t sound right! And the same applies to dev-java/jna (which seem to provide similar features, by the way). The problem with both is that they really should have a way to tell at build time were their native libraries should be found, and then leave the property as a way to override it… Unfortunately it seems like there is no way to do that. I reported a feature request for JFFI and we’ll see how it’s going to turn out.

Unfortunately, the problems don’t stop here. Another problem happens with the OpenSSL-compatibility extension provided by dev-ruby/jruby-openssl; the idea is to provide the same interface of Ruby with the OpenSSL extension, but using Java libraries instead. Since the standard JRE does not provide all the needed interface, this extension uses BouncyCastle to do the work, and that does not work fine here at all: not only the extension bundles a copy of bcprov and bcmail (even in the very git repository!), which by the way don’t seem to work right for me, maybe because they are the JDK 1.4 versions, while we use at least Java5, if not 6 altogether, but more importantly our already-present bcprov/bcmail packages fail badly. The issue here is that bcprov can be loaded as a JCE provider; JCE providers need to be signed, and obviously our rebuilt Jar file is not signed. This is a huge problem that the Java team should most likely look into and something I really cannot look into myself.

On the bright side, though, tonight I’m going to commit a new JRuby ebuild with a much simplified launcher script (and no more symlinks around!). This basically cuts all the conditionals for cygwin, as well as for dynamic discovery of JAVA_HOME path and replace them with assumptions about the Gentoo setup (JAVA_HOME is filled in during install phase) and calls into java-config. Hopefully, this should reduce both possibility of mistakes, and reduce the time needed to process the script. Unfortunately, the script as it is, uses bash, that we all know is far from the fastest shell out there. Porting it to use pure sh is definitely not possible — although one could argue that Java’s own startup is definitely going to take up more time than using bash).

Oh well, we’ll see how it goes from here.