Test comprehensiveness versus replicability — A Ruby Rant

I guess that “A Ruby Rant” could become a regular column on this blog, given how many of my posts over time has been “Ruby Rants”, but let’s not dig further.

I’m trying my best to package the new dependencies introduced with Radiant 1.0.0rc4 so that we can update the package in Portage (given that Radiant 0.9 is in tree already). This is proving quite difficult; even though Radiant upstream helped me out by replacing the old dependency on highline with a modern one after the issue was fixed upstream, there are a few new gems that require hours and hours of work to package.

The first big issue comes with cocaine — this gem is developed by ThoughtBot, which are the Rubygems.org designers, one would expect highly professional development from them, but that’s by far not the case. The gem requires another ThoughtBot-developed library for testing, bourne which in turn requires mocha, but not any mocha, up to yesterday it required strictly version 0.9.5; now it requires strictly version 0.10.4, which is an improvement but still not kosher. Why this happens?

@flameeyes It hooks deep into mocha and breaks on internal changes. We’d love an internal api for mocha, but there isn’t one.

So here you can see one huge issue with Ruby development: since it’s very hard to actually make internal interfaces internal, due to monkey-patching and scope games, people will end up relying on things that they shouldn’t be relying upon. And they pretend that’s a good way to solve the issue because there is not a better way. Damn.

Okay strike one, I have put cocaine aside for now (and that was a good idea, seeing how prodding them about it have gotten them to at least update bourne’s dependency on a version that is not quite as ancient), and worked on another dependency: compass which is yet another CSS framework.

Testing this particular package has proven quite difficult, because it has a long list of extra dependencies that you won’t find listed in the gemspec at all, but just on the Gemfile. This list among others include compass-validator which is a gem only ever required by compass, which requires compass on its own (without listing it in the gemspec)… even if this sounds fishy enough it doesn’t cover how fishy that gem is.

The compass-validator gem bundles a series of prebuilt Java libraries up to W3C’s CSS Validator and executes Java at runtime. No, this has nothing to do with JRuby, and even when using JRuby it wouldn’t load the Java classes but still execute Java. Thanks to Krzysztof at least I’ve been able to get the gem to not bundle so much Java code; instead it relies on dev-java/css-validator and java-config to call into the right commands. Still nasty but at least usable.

But this is not yet a nasty problem: the Gemfile lists autotest which is a dummy gem for ZenTest instead (so why not list ZenTest directly? bah!), and then proceeds with two different ways to wrap around gems specific for OS X: autotest-fsevent is under a RUBY_PLATFORM check, while rb-fsevent is under a group. The problem is that Bundler always install all the groups, despite not loading them if you exclude them.

Okay so the gem has some specific codepaths for OS X, does that matter much? Probably not as long as you can still ignore them and run them on Linux. But there is more trouble ahead when I reach the terrific livereload gem, which is described this way:

LiveReload is a Safari/Chrome extension + a command-line tool that: 1. Applies CSS and JavaScript file changes without reloading a page. 2. Automatically reloads a page when any other file changes (html, image, server-side script, etc).

Okay so it requires Chrome at test-time.. not so surprising as another of the gems I use (best_in_place) indirectly uses selenium-webdriver, which uses Firefox. The problem is that when you go to the source repository for that gem, you’re told that it’s deprecated (compass is still actively developed!) in favour of … a graphical Mac/Windows application.

So okay I’m all to cover as much as possible with tests, but how does that help if you make it impossible for anyone else but you to actually run the tests because you tie them to one specific platform, and one that is very unlikely to be used as production server?!

Sigh.

Exit mobile version