Seamless access to files, IO streams, compressed files, …

Since the testsuite for Ruby-Elf starts being disproportionate to the actual code that Ruby-Elf consists of, and I’m still lacking regression tests for the two scripts in it (missingstatic and cowstats), I’ve considered some time ago to support accessing ELF files compressed with either gzip or bzip2 so that the space required would be drastically reduced.

Unfortunately my idea ended up being unrealisable at least at the time since Ruby-Elf needs to seek, and neither formats allow for easy seeking around.

I started working on some generic IO access to files in a branch, but it didn’t turn out very good, and I left it behind for a while. Since now I’m at the point I really need to write the testsuites for the two scripts, I decided to revive the idea, and implement it with a system of “backends”.

I started with two simple backends: access through path (with a File instance) and access with a direct IO stream. Very easy and not really complex at all. Then I introduced a ruby-mmap backend so that the file could be mapped into memory rather than read and copied over, and this also was fine, although I had to emulate the cursor handling (seek and tell). Reading gzip compressed files was also quite easy since Ruby already provides a good interface to that. Unfortunately bzip2 support is a totally different matter, since the bzlib interface does not provide the tell and rewind methods that are needed to emulate seek for compressed files (slowly).

The problem at this point is that the code for the backends is complex on its own, and it would add over Ruby-Elf’s complexity to a point that they wouldn’t really make sense together at all. Reached this point, you know what it comes: factoring the code out.

For this reason I’m now thinking what the best course of action can be; I want to have access to possibly a lot of backends: straight files, compressed files (gzip, bzip2, lzma), archive files (tar and pax, ar, zip, … — ruby-libarchive would help here), network files and so on. I already decided on the name: ruby-bombe (if you like history you should get the reason for the name), the problem is now taking the code out of Ruby-Elf, write ruby-bombe, adapt Ruby-Elf to the new library, and hope none of the three users of ruby-elf gets mad at me for requiring a dependency.

Tonight is going to be a long night.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s