Gentoo and EPSON scanners

“Harry,” Susan said. “Have you ever heard of the paperless office?”

“Yeah,” I said. “It’s like Bigfoot. Someone says he knows someone who saw him, but you don’t ever actually see him yourself.”

Jim Butcher — Changes

Because of the way Italian bureaucracy is designed, most of the business communications I deal with are still in paper form. And since my business lately has been covering a number of different customers, with different degrees of formality, I’ve been pretty much swamped by paper. I have been looking for ways to reduce the amount of paper I have around, or at least the amount of paper I have to look at often enough, and the solution was that to get a scanner with auto-document feeder (ADF) and keep all the new paper document scanned before either shredding or archiving them.

I settled first for an HP OfficeJet multi-function printer (which jammed so many times it wasn’t funny, but at least was cheap) and then for a LaserJet one that I was given by a family friend who couldn’t use it via USB any longer. Unfortunately it was a M1522nf, which turns out has a defective formatter card which requires either baking or replacement. HP wouldn’t replace it without paying a couple hundreds of euro, which wouldn’t be worth it at all.

I went to look for a solution, with the understanding that my current laser printer (a Kyocera-Mita FS-1020D) is quite a charm to use, especially now that it is no longer connected to an Apple Airport Extreme, but rather to my Gentoo-based router whose cups instance is directly referenced by both Yamato and Raven so that I don’t have to run it on them any longer. I went to look at more professional, office-oriented scanners, since the only one I could find in “entry level” by HP listed a duty cycle of 100 scans/day (which I’m sure I would bust easily; a bank-related contract is usually longer).

The requirements, beside being able to scan a decent amount of sheets (more than 20) were generally the usual you’d expect for a person like me: it has to work with Linux, possibly 64-bit, with preferably no prebuilt, proprietary software. To be honest, this doesn’t seem to be feasible at all. And before somebody asks, no HP is not much better. Sure HPLIP is open source and Free, but to use the scanner in the aforementioned M1522nf, you have to install their proprietary plugin bits. Canon scanners, which seems to be the cheapest with Linux support for ADF, are supported by their driver, but it’s totally proprietary and more importantly it only works on 32-bit systems.

EPSON, instead, seems to have a more interesting approach. The Avasys-developed epkowa backend for sane is actually quite nice: it encapsulates a decently-sized open-source backend with a number of proprietary plugins (and in some cases firmware files). I already had a bit of experience with that backend before, since my flatbed scanner is a Perfection 24802580, but with that I couldn’t make use of it in a long time simply because the non-basic functionality (film scanning) is supported by a 32-bit only plugin.

After snooping around and poking my hardware supplier about it, I decided to get a GT-S50 (sheet-fed) document scanner, which is shaped more like an inkjet printer or a fax machine than a scanner. This time as well a plugin is needed, so based on the GT-F720 ebuild that was already in tree I wrote one for the required plugin (and also one for the GT-F500 plugin that is used by the Perfection above — but the only task I can use it for is to install the firmware file used by snapscan, so if you want to try it on 32-bit please let me know if it works at all!). You can now find iscan-plugin-gt-s80 (it includes support for both S50 and S80) and iscan-plugin-gt-f500 in tree already.

I got the scanner on Friday evening and set it up; the driver worked at the first try, and the iscan tool provided by Avasys/Epson works just right. On the other hand, my customized script over scanimage (as provided by sane-backends) was not working properly when scanning duplex (i.e. both sides of the sheet at once). In such a configuration, a single pass of the sheet in the ADF causes two pages to be read; the driver (or the firmware, not sure) caches the back side as the second page, and the next time sane is asked for a page, it returns the one already scanned. When doing the second scan, it was random whether it worked or not: a race condition.

I haven’t spent enough time on it yet to know where the problem lies exactly: it might be a bug in the scanimage frontend (given that iscan works fine), or a bug in the drivers. At any rate for now I worked it around by adding a sleep(1) before calling sane_start() in the file. You can find a hacked 1.0.22 ebuild for sane-backends in my overlay if you happen to have similar problems.

At the end of the day, I’m pretty happy with the device, it scans very nicely, very fast and has a high enough duty cycle for the kind of stress I could make it go through. Once I’ll be able to resolve the above-noted issue with scanimage it’ll work much better. But there are a few issues that still need to be solved, such as:

  • iscan needs to be set to use /var/lib/iscan rather than the current /var/lib/lib/iscan, and that requires plugins to be re-registered;
  • I need to find a way to register the plugin properly when installing in a different ROOT, which right now neither of my ebuilds do;
  • the iscan-plugin-gt-f720 ebuild needs to be brought up to speed with the other two, as right now it seems sub-par (among other things it installs the plugins in /usr — which, albeit being what AVASYS does on RedHat systems, is wrong, as they should go in /opt).

Frontends to command-line or libraries?

I know I’m still convalescing so I should be resting, not thinking about development problems, but this is something that ended up in my mind because I have one thing to absolutely do (scan the release documents from the hospital to send to my GP), and for which I miss an easy interface I could instruct my mother, or my sister, to use.

Don’t get me wrong, I know there are a few SANE frontends, starting from xsane itself, but the problem is that I don’t usually stop at scanimage when I do it from the console. What I usually do is to launch a batch scan to TIFF format, then use tiffcp to join the different TIFF files in a single file with multiple pages, and then use tiff2pdf to convert it to a PDF file, which is opened by any computer I might need to send the data to (and it also is quite smaller than the original TIFF file). Lately, I’ve started trying to add to the chain also unpaper, a tool that removes some of the defects usually found in scanning pages from books (like black borders and similar), which works on PNM (thus requiring a change in the scanning command, and a further conversion later on).

But I don’t want to start fleshing down how to actually write such a tool, or I might actually start writing it right now, which is not what I’m supposed to do while I’m convalescing.

What I want to think about is that here comes one huge debate between writing frontends for command-line tools, which just interfaces with them as process calling, or writing a full-fledged program that interfaces with the low-level libraries to provide similar functionalities.

I already happen to discuss that quite often since xine and mplayer embody the two spirits: xine has its library, frontends interface with that, and a single process is used; mplayer instead has a command interface, and frontends execute a new process for playing videos.

There are of course advantages and disadvantages, one easy to spot disadvantage to xine’s approach is that a crash or freeze in xine results in a crash or freeze of the frontend, which is something Amarok users have been unfortunately familiar with.

In the case of the scanning toolchain, though, I guess it’d be probably easier to use a frontend for the tools, as re-implementing all the different functionalities would be a non-trivial work.

The disadvantages of doing it this way, though, is that you’ll have to make sure that the tools don’t change their parameters between versions, otherwise it’d be a problem to ensure the correct functionality of the tool. Also, the tools need to provide enough options to control with granularity the execution of their task, which is something unpaper actually does, but in turn makes them almost unusable for final users.

I know I’m not going much anywhere with this post, I’m afraid, but I just wanted to reflect on the fact that to have a command line tool designed to be used by frontends, you almost certainly make its syntax so complex that users would fail to grasp the basic concepts, and in turn, you’d need a command line interface to the tool too… which is why there are so many scripts interfacing to ffmpeg for converting videos, I guess.

On the other hand, one can easily write such a frontend using scripting languages, even if it’s graphical, such as with ruby-gtk2 (think RubyRipper).

You know that your health took a bad turn when…

… your medical records folder is ten times thicker than the job contracts folder. I was cleaning up through the paperwork yesterday and today, and there are so many things.. Luckily I have most of my CT scans in digital format, beside the last one I did at the ER last week, and a cerebral CT from a few months ago. But the release paperwork I had only in printed form, which I needed to scan and “PDFize” on Tuesday to send to my GP — nice to have one who’s reachable via e-mail.

It’s the scanning that actually made me think a bit. I have the scanner a bit far from my workstation; to scan a five pages document I have to prepare the scanimage command in batch mode on the workstation, then walk all around my desk to get to the scanner, and then get my arm around the monitor to press the return key on the keyboard to start the scan.

The annoying thing is that the scanner has four buttons on it, that should be made just for the task of starting the scan. Unfortunately these buttons don’t work out of the box on Linux at all. There is a package, called scanbuttond that polls for them through libusb and then execute a custom script when they are pressed. But as you can guess, polling means it uses a lot of CPU, and the fact that they run a generic script makes it less integrable in a desktop environment. Of course it would be easy to port scanbuttond to submit the read buttons back into the kerel input subsystem so that they appear as generic events, but… I think this should be a task well suited for a kernel module, hooking them up directly in the input subsystem, so that evdev could pick them up and then a program could just wait for them as shortcuts to have some action.

I tried looking into writing something before, but I ended up stuck in a problem: would a kernel module interfacing with the scanner interfere with libusb access by sane itself? Last time I enquired Greg KH, he asked me to proivde lsusb -vv output but, sorry Greg, work piled up and I forgot about all of it (until yesterday when I had to scan some more documents). Well, if anybody wants to take a look, this is it for my current scanner:

Bus 001 Device 003: ID 04b8:0121 Seiko Epson Corp. Perfection 2480 Photo
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x04b8 Seiko Epson Corp.
  idProduct          0x0121 Perfection 2480 Photo
  bcdDevice            0.00
  iManufacturer           1 EPSON
  iProduct                2 EPSON Scanner
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           39
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           3
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered

Can anybody tell if it would be possible for a kernel module and sane to access it at once? :)

If there is the chance, I might look at it once I feel better. I haven’t written or touched a kernel module in so much time that I’m feeling like doing some work in that regard. And the code to actually get the data out of the scanner should be present in scanbuttond already, it’s just a matter of getting it “pushed in” rather than “polled for”.

Once I could get the buttons working, I’d probably be working on a GTK-based frontend for scanimage with handling of those, so I could just use those rather than having to set it up manually. Although the nice thing of scanimage is that, through that, tiffcp and tiff2pdf, I can quickly create a multi-page PDF of the scan (the first command creates a multi-page TIFF file, the second converts it to PDF), and if I do the scan in lineart (the perfect solution for B/W text) it also is tremendously small in size. I should try to have the same result with my frontend. My idea would be a light frontend written in Ruby, calling the commands directly.

Oh well, at any rate, this will have to wait till I’m feeling really better, or maybe it’s just unfeasible because of the way it’d need to access the scanner from kernel and libusb.

I should retire…

… and I mean in a small house in a small town in the alps, far away from computers, ADSL connections, and all these things.

Why? Well, if you follow my blog you know I had some health problems this summer, and the main suggestion of the doctors was trying to relax more. As it was, I really didn’t relax at all, Gentoo and xine and other Free Software projects sucking up all my time.

Now, today I was supposed to have a job interview, but in the last couple of days my blood pressure was a bit too high, and my doctor told me to just relax for some days, this included cancelling the meeting. Of course I need to keep controlled the pressure because I also have the warfarin to take, and I need to get my INR tested again, too.

So okay, after passing the morning reading, and finding out that this just make me think too many times about my problems, I decided to try my luck today by trying to learn how to write a Linux USB driver. Well, by trying again at least.

Last time I tried it was to see if I was able to implement a driver for a USB to serial adapter that I bought on eBay: out of four of them, three were PL2303 (perfectly working) and one used a chipset (a Chinese producer) that is not supported by Linux, and I failed to find any specification for it. I tried more than once to contact the producer, with no results. Sniffing the interaction between Windows and the device I was able just to find some commands. but nothing useful to actually make the device usable.

This time, I started thinking about the four buttons on my Epson scanner. The only thing I found to actually make them useful under Linux was the scanbuttond project, but I don’t really like its approach. The first problem is that the daemon polls for the status of the buttons, and this isn’t really a nice approach for the CPU. The second is that it executes commands, which aren’t really well put into context, and especially you can’t use it to provide commands to a scanning application.

The Linux kernel provides a very good way to handle buttons of any kind, the event devices. Even the ACPI buttons are now event devices, and those are easy to take into context: just make sure the application opens the correct event device, and that should take care of reading the buttons.

So I said to myself: “why can’t I try to write a driver for those buttons that shows the scanner as an event device?”. This time there was nothing to reverse, the code was written already; in userspace using libusb, but it’s there as a reference. And I started.

I almost forgot having a copy of Linux Device Drivers, it arrived not long before I ended up in the hospital, and I tried to remove most of the last weeks before that, as well as the first weeks in the hospital. I’d like to remove the whole hospital experience, but that’s near to impossible.

So I started reading the USB part, and I was able to come to something that at least loaded and identified the scanner. I did make some mistake on the disconnection function, as my kernel paniced; I restarted with the intention of using VMware to test it next time, at least then I panic just the guest OS and I can restart it without stopping my work.

But when I restarted I also read the reply to the mail I sent to Daniel (dsd) before, and he gave me the bad news: if the interface I need to read the buttons data from is the same that SANE needs to access the scanner, then it won’t work anymore. And of course, Epson didn’t use different interfaces, it’s just the same. Different endpoint, but still same interface.

So I’ll probably just rm -rf the code I wrote up to now, as it makes it useless… it can turn a scanner into a 4-buttons keyboard, but I doubt this is what users want.

Seems like this is just another failure for me…

Update on usb.ids

I already wrote before about my intention of taking over usb.ids, and I’ve uploaded a copy of a modified one at Today I updated it with two entries sent to me by Waldemar de Laurent and Sebastian Siewior. Remember that if you have any USB device that is not detected by my usb.ids (and thus even by the official one) you’re very welcome to send to the USB IDs for it.

I also added a lot of EPSON scanners, fixing a few entries that have the same IDs pair for multiple models (because of different options). The entries are taken by the udev rules file in media-gfx/iscan package (that is listed by EPSON itself).

I think there are other similar files out there I should be looking at to update usb.ids with newer values; sigh.

Update from the Java front

So, yesterday I blogged about Free Java implementation Kaffe, and today I have some updates on that.

First of all, dalibor on #kaffe suggested me to try CVS head, and of course it make sense, considering the fluent state of the project, so I added a CVS live ebuild to my overlay, keyworded -*.

The he explained that the problem is with GCC 4.1 that changed something in exception handling, and provided me a patch to try out, that actually worked. The main problem is that it forces -O0 to build.

After patching Kaffe, and building it with those totally un-optimised flags, the result is quite good, as Yahoo games works fine, albeit quite slow, and the CPU usage is about 70%. This makes Kaffe un-optimal for people relying a lot on java, but make it good enough for me that I don’t use it much a part from playing literati from time to time. Just a note: you need to disable KIO support in Konqueror or loading java applets will fail.

Following this Java improvement, I have to thank sbriesen for helping me setting up my Epson scanner; it seems like I need only the firmwares out of media-gfx/iscan, and then snapscan backend works fine. Yuhuu.
Okay now I can actually reduce my OSX usage to the minimum, as to do copies I can just use Kooka. Unfortunately I wasn’t able to find an application (yet) that is able to read the negatives out of the scanner and split them in three different photos as it’s done by Epson’s utilities. I’m not a graphic person, so I have no idea where to start, and thus I can’t help with it, but I’m ready to test anything if someone is going to work on such an application.

So now the other main binary thing I rely on is the nvidia driver, that’s going to take the way of the sea next week most likely, when my 9250 will come. €55 might e a good price for Freedom. I’ll tell if I have any quirk with that card that might be needed.

Oh yes and I (with a bit of luck rarely) rely on win32codecs for some wmv9 files, but that’s not going to be a problem as soon as FFmpeg will support VC-1 codec, so I’m not even considering that an issue, for how little times I actually need it.