Ruby-NG: Code of Horror

I’ve decided to try basing the titles of the blog posts about Ruby-NG, from now on, on the titles of Star Trek: The Next Generation episodes to have some fun myself when I post this (quite unfun) content. Hope you like the idea.

Even if some people seem to try painting me as an anti-Ruby activist, I’d like to repeat that I like Ruby as a language, and to some extent I like Rails as a language. What I don’t like is the way that a lot of things in the Ruby environment seem to be doing: make it easier to create low-quality code, and make use of it. Don’t get me wrong: there is a lot of good Ruby code out there, and it’s often easily available because of tools like RubyGems, GemCutter, RubyForge and, increasingly, GitHub. The problem is that while these tools make it simple to produce and distribute good code… they also make it too simple to produce and distribute bad code.

I have ranted many times about the problems related to testsuites: while Ruby, and Rails in particular, seems to have been considered by many the apotheosis of Test- and Behaviour-Driven Development (TDD and BDD), and of the Agile development model (which also stress testing at both unit and integration levels, although not as much as the other two), most of the software that gets published for these environments don’t seem to apply it all too much. Even when tests are present, they might fail because of blatant errors in the code itself, or because of wrong assumptions about the standard provided by the language and other libraries the current code makes use of. And even when they are written properly so that these problems don’t arise, they might not be tested to work outside of the development repository of a package, with missing test sources, or missing help files that are needed for the tests to succeed.

Tight integration between different projects of the same developers also tend to be quite a problem when trying to independently verify the tests handling: this is the case for most of the software that is released by the seattlerb project that expects the sources for all the different extensions to be present in the same directory, and unversioned, for some of the testsuites to properly execute. A similar problem arose today with rspec-rails accessing rspec directly in its testsuite; I have to report this upstream tonight.

Another common problem caused by the very simplified distribution and handling provided by RubyGems is the easiness with which many projects break their interface, or with which other projects use interfaces that are not guaranteed to be stable among releases. In the first case, the solution is to hope that all the dependencies are written correctly, and thus that slotting the package will not break anything. For the other case, the problem is much bigger. Projects like those keep on being used because there is an implicit slotting of all packages by default in RubyGems, so if you depend on a particular version, say test-unit = 1.2.3 you generally won’t trigger any suspicious behaviour by RubyGems. But it certainly is frown upon by distributions like Gentoo, to depend on an older major version of a package, whatever package that is.

And again it also doesn’t help that the various Ruby versions and implementations have changed the amount of software bundled with them: Ruby 1.9 dropped test-unit (now available as a standalone Gem, which is not working properly with Ruby 1.8 or with RSpec) and instead added minitest (which is available standalone for Ruby 1.8 and JRuby). RDoc that was once standalone, then it was bundled with Ruby, and again now a new version (RDoc 2) is released as a standalone gem (although hanna works only with an older version as it depends on some internal interfaces that have, since, changed a lot). JRuby comes with its version of rake as well.

Don’t get me started on the various middleware packages used on top of RubyGems either: Hoe, Echoe, Jeweler. Three options to do more or less the same thing (testing gets even worse, as I said above, but I’ll pick it up in a moment); if you’re lucky, the developers know enough to make them optional, and simply omit the gem-publishing tasks from the Rakefile when they are not available (it might be bugged, like it happened on samuel, but it’s easy to fix in that case); in other cases you’re not so lucky so you have to depend on them, for both building the documentation and running the tests, or even building the extension if it’s native code. Up to now Gentoo has been able to ignore the necessity of Jeweler, but we ended up having to keep around Hoe and Echoe at least — they both look more widely (mis)used: I don’t think I have encountered any package that mandates Jeweler, if not for a minor bug.

Returning to the above noted “there are many ways to skin a cat” (I find this phrase icky, but I don’t know of a nicer alternative in English, so if you have suggestion, they are welcome; I’ll also gladly edit the post), testing is another quite not-so-nice situation. While I can understand that different testing frameworks have different targets, and everybody should be able to use whatever tool they are comfortable with, this adds quite a burden over distributors, and generally people wanting to verify the software they depend on. It’s not just a matter of finding which packages are needed for tests (as most packages don’t seem to document that at all, neither in gemspec nor in the documentation), but also to find how to launch them: it might be rake test, rake spec or even rake examples; sometimes, two or more of those options are available, but only one is the right one, so it requires human intervention and similarly happens to the API documentation: rake rdoc, rake redoc, rake doc, rake rdocs, …

I already tried prodding other distributors into authoring a set of guidelines (not rules, but suggestions welcomed by a wide and varied group of developers) for packaging of Ruby extensions. I’ll probably get back to that at some point (for now I have other projects that I should be taking care of). Lacking a mandated interface for RubyGems, we can at least try to write down some suggested interface – like how to call the tasks for testing and for API documentation generation – and maybe convince the various middleware packages named above to implement the equivalent of make distcheck for autotools: package the extension, then extract it in a different path, and run the testsuite; if it fails there is something wrong. Of course this also requires to convince the developers to use that feature.

And if somebody will complain now that this would be bad for the Ruby community because “it limits the developers’ creative freedom”, then they have no idea what it means to develop complex software, and especially to make it work as intended.