Dealing with closed-source firmware and latent bugs

Another day, another customer with issues with the closed-source firmware of one of their devices, even if this time it’s not as bad as the one who got angry yesterday when I dared suggest I would have liked to see the device’s firmware to see if I could isolate the bug.

This time the device is a RIP server for Xerox printers, developed by EFI (no, not the one Matthew Garrett is fighting with). In particular, it’s the server that comes with a DocuColor 250 “copier” a very old customer of mine has recently acquired.

This copier, which is actually a much higher end multi-function printer, is replacing, for my customer, the previous DocuPrint 700 copier, which also has an EFI RIP server. The 700 is actually the more recent one, but that’s my customer’s choice to go with an older version, and thus I have no say in the matter.

As you can probably guess, both devices come with a scanner (otherwise they wouldn’t be called copiers), and both scan to so-called “mailboxes”, which are on-RIP storage areas with predefined configuration values. One of the most important things to keep in mind with these mailboxes, though, is that they can be set to transfer the scanned files to a different storage area through the network. On the DP700 RIP, I was able to set it up to use CIFS to send the data on a shared folder on their backup server – after they updated to OS X Lion, since Apple replaced Samba with a SMB2-only server – but the same trick wouldn’t work with the DC250, since that version, much older, does not support CIFS at all.

Instead, to push files out, this version only supports FTP, which is generally nice and easy: just set it up on the same server, to access the same shared directory, and you’re done.. or not? Turns out that the release notes for the RIP firmware state explicitly this:

Scan to FTP not supported for all FTP servers

Scan to FTP is not supported for all FTP servers. If an FTP server does not reply to a NLST command with a 4xx or 5xx response, scan to FTP from the Fiery does not work correctly. The WS_FTP server is known to have this issue.

While these three lines are all the documentation you can find about the topic, a quick sniffing around gave me a good idea of what the problem is. When writing the files to the FTP server, the RIP software is trying to find a unique name to use; instead of using the current date/time to generate one, it tries over and over until it finds a free one. The problem is that to do so, it expects the NLST (short listing) command to return an error if there is no file matching the name… which is not what RFC959 (the specifications of FTP) tells you to do. VSFTPd does it exactly like WS_FTP is said to do above: it will return a success message.. and an empty list.

Again, this is the kind of problems that I would like to be able to fix, but since this is a closed-source, proprietary firmware, it’s out of the question that I’d be able to (besides, the developers know the problem exists already, as they wrote it down, yet there is no fix available). Combine this with the obnoxious experience of “local” developers who pretend to be holy and get angry I dare to think I’m as good as them, and you can probably guess why I’m a Free Software developer.

The web application security culture

Okay, I love to rant, so what?

Just the other day I have complained about Rails’s suggestion for world-writable logs and solved it by making it use syslog and now I’m in front of another situation that makes me think that people still don’t know how to stop themselves from creating software that is pretty much insecure by design.

So what’s up? For a customer of mine I ended up having to install a full LAMP stack, rather than my usual LAPR. In particular, this is for a website that will have to run WordPress. Thankfully, I have ModSecurity to help me out, especially since not even two hours after actually setting up the instance, Spiderlabs announced two more security issues including an extract of their commercial rules.

Anyway, the WordPress instance will have to be managed/administered by a friend of mine, who has already had some trouble before with a different hoster, where the whole WordPress instance was injected with tons of malware, so was quite keen on letting me harden the security as much as I could… the problem here is that it seems like there’s not much that I can!

The first problem is that I don’t have a clean way to convert the admin section to forced SSL: not only wp-login.php is outside of the admin subdirectory, but most of WordPress seem to use fully qualified, absolute URIs rather than relative URLs — such as the ones I’m used with Rails, which in the case both of Typo and Radiant let me restrict the admin/ directory to SSL quite easily. Why is that so important to me? Because I would have used an admin URL outside of the website’s domain for SSL: I don’t own a certificate for the website’s domain, which is not mine, nor I want to add it to the list of aliases of my own box. Oh well for now they’ll live with the “invalid certificate” warning.

Next stop is updating the webapp itself; I was sure at that point that “updating the webapp” meant letting the web server write to the wordpress deployment directory… yes, but that’s just part of it. As it happens, plugins are updated via FTP, like my friend told me.. but not in the sense of “downloaded from an FTP website and written to the filesystem” but the other way around: you have to tell WordPress how to access its own deployment via FTP. In a clear-text web form. Admittedly, it supports FTPS, but it’s still not very funny.

I’m unsure if it was a good idea on my part to accept hosting WordPress: we’re talking about installing MySQL, PHP, vsftpd and enabling one more service on the box (vsftp) just to get a blogging platform. Comparatively, Rails look like a lightweight approach.

Updating to 6.1_rc1

So, since yesterday I worked on updating Gentoo/FreeBSD ebuilds to 6.1. Thanks to Robert (arachnist) the work was quite easy, as he already tired most of the build stuff :)
What remained me to do was to fix the details and making sure that the result was fine, along with trying to build the stuff with SSP enabled.

Unfortunately I forgot (entirely) freebsd-usbin yesterday, and I’m working now to fix it. This wasn’t good mainly because the two applied patches failed, one has been fixed by shortening it and removing the hacks to use /usr/src/sys (as thanks to Benigno we now symlink the src directory in ${WORKDIR} and live happy), the other I had the patch from Robert to replace with.

That wasn’t enough, as it then failed on amd’s info file… but that made me notice that amd came out of contrib package… and that am-utils package is in portage. I’ve then removed amd from freebsd-usbin and now I’m fighting to make am-utils to build on Gentoo/FreeBSD. The main problem is due to the different way db is handled. On GLIBC, to have db1 you need to install it separately, but in FreeBSD the functions are provided by the libc, so you need nothing to link against, but at the same time, things like gdbm are more likely to conflict. In this case am-utils are picking up the ndbm.h header from gdbm but no library :/

I should be able to get that to work later today, in the mean time luckily I don’t use amd at all :P So next time the stage will be even smaller, as amd won’t be present (let’s minimal be minimal, do we?)

What remains in my mind as “to be decided” is the fate of ftpd and lukemftpd. lukemftp (the client) is removed in favour of tnftp (that’s the same code but updated with a new name), but we don’t have tnftpd (the server) in portage, although we do have quite a few of FTP daemons.
I’m thinking of simply dropping those two ftpd and leave the users with the choices already in portage, if someone really wants a lukemftpd-compatible daemon, I’d rather put tnftpd in portage independently so that it can be used by Gentoo Linux users, too, and it’s updated more regularly.

For who’s wondering yes, lighttpd here is already running with SSP enabled, as well as ruby :)