Paperless home, sorted home

You probably don’t remember, but I have been chasing the paperless office for many years. At first it was a matter of survival, as running my own business in Italy meant tons of paperwork, and sorting it all out while being able to access it was impossible. By scanning and archiving the invoices and other documents, the whole thing got much better.

I continued to follow the paperless path when I stopped running a company and just working, but by then, the world started following me and most services started insisting on paperless billing anyway, which was nice. In Dublin I received just a few pieces of paper a month, and it was easy to scan them, and then bring them to the office to dispose of in the secure shredding facilities. I kept this on after moving to London, despite the movers steaming my scanner, using a Brother ADS-1100W instead.

But since the days in Italy, my scanning process changed significantly: in Dublin I never had a Linux workstation, so the scanner ended up connected to my Gamestation using Windows — using PaperPort which was at the time marketed by Nuance. The bright side of this was that PaperPort applies most of the same post-processing as Unpaper while at the same time running OCR over the scanned image, making it searchable on Google Drive, Dropbox and so on.

Unfortunately, it seems like something changed recently, either in Windows 10, the WIA subsystem or something else altogether, and from time to time after scanning a page, PaperPort or the scanner freeze, and don’t terminate the processing, requiring a full reboot of the OS. Yes I tried powercycling the scanner, yes I tried disconnecting the USB and reconnecting, none seem to work except a full reboot, which is why I’m wondering if it might be a problem with the WIA subsystem.

The current workaround I have is to use the TWAIN system, which is the same that I used with my scanner on Windows 98, which is surprising and annoying — in particular I need to remember to turn on the scanner before I open PaperPort, otherwise it fails to scan and the process will need to be killed with the Task Manager. So I’m actually considering switching the scanning to Linux again.

My old scan2pdf command-line tool would help, but it does not include the OCR capabilities. Paperless seems more interesting, and it uses Unpaper itself. But it assumes you want the document stored on the host, as well as scanned and processed. I would have to see if it has integration with Google Drive, or otherwise figure out how to get that integration going with something like rclone. But, well, that would be quite a bit of work that I’m not sure I want to do right now.

Speaking of work, and organizing stuff — I released some hacky code which I wrote to sort through the downloaded PDF bills from various organizations. As I said on Twitter when I released it, it is not a work of engineering, or a properly-cleaned-up tool. But it works for most of the bills I care about right now, and it makes my life (and my wife’s) easier by having all of our bank statements and bills named and sorted (particularly when just downloading a bunch of PDFs from different companies once a month, and sorting them all.)

Funnily enough, writing that tool also had some surprises. You may remember that a few years ago I leaked my credit card number by tweeting a screenshot of what I thought was uninitialized memory in Dolphin. Unlike Irish credit card statements, British card statements don’t include the full PAN in any of the pages of a PDF. So you could think it’s safe to provide a downloaded PDF as proof of address to other companies. Well, turns out it isn’t, at least for Santander: there’s an invisible (but searchable and highlightable) full 16-digit PAN at the top of the first page of the document. You can tell it’s there when you run the file over pdf2text or similar tools (there’s a similar invisible number on bank statements, but that’s also provided visible: it’s the sort-code and account number).

Oh and it looks like most Italian bills don’t use easily-scrapeable layouts, which is why there’s none of them right now in the tool. If someone knows of a Python library that can extract text from pages using “Figure” objects, I’m all ears.

Packaging EPSON’s scanners’ drivers

You might remember that a couple of years ago, I bought an EPSON scanner – a GT-S50, no longer sold, replaced by the GT-S55 – to scan invoices and other kind of documents that I had laying around. The investment paid off, in my eyes, because now I have all my documents digitized, always available, and I was able to fill quite a few bags of shredded paper. The amount of work involved in digitizing that many documents without a professional scanner would have costed me in time much more than the device costed me in money.

But as I noted in the blog post above, to get the scanner to work on Gentoo Linux, I had to create ebuilds for a new package with the proprietary plugin that can handle the scanner — it’s not a sane plugin per-se, it’s rather a plugin for the EPSON-provided epkowa backend, which is otherwise opensource. I actually ended up adding two packages to the tree to be able to do that.

Nowadays, the media-gfx category hosts a number of plugins for epkowa, under iscan-plugin and esci-interpreter prefixes — the latter is the case for the GT-S50; as far as I can tell, the main difference between the two is that the latter ships with no firmware, but just a protocol interpreter, as the name suggests. Together with this, I maintain another ebuild for the plugin (32-bit only) and firmware (32- and 64-bit) for my older Perfection 2580 scanner, plus two more packages that I proxy for users who can actually test on good hardware. Today I even added another package for a scanner I don’t own, but I plan to buy when I settle down in my new residency (the 2580 stays here with my mother), the Perfection V370 which is the first one in our tree that actually uses “perfection” in the RPM name.

The ebuilds are almost identical to one another, but there are quite a few tricky bits around them. First of all, we don’t install in exactly the same locations as the RPMs — the reason for this is that they use /usr as base path for everything, but in our case, since these are provided by an external source, and are pre-built, they are installed in /opt instead, with the exception of the firmware, that stays in /usr/share simply because that’s where some of the software looks for it (namely, snapscan backend for SANE).

Some of the plugins are valid for multiple USB IDs, as is the case for the GT-S50/55/80/85 interpreter, while others are only valid for a particular ID – which may refer to multiple models anyway, as sometimes it refers to the main controller on the device, which may be shared — in the case of the 2580 noted above, it shares the same flatbed with the 2480; the difference between the two models is all in the lid, in both it hosts the lamp for scanning film, but in the 2580 it also includes the motor to load the film semi-automatically.

You can easily see which IDs the plugin applies to by using strings on the rpm and look for iscan-registry — but at this point question should be: where do you get the RPM? Well, unfortunately AVASYS no longer provides the download at a known location; EPSON themselves are distributing it, through Akamai — which means that we can’t point to the upstream servers anymore, and euscan won’t be able to find a new version being released. To solve this situation, the solution we opted for is to mirror the rpm files on my webspace on dev.gentoo.org including the needed documentation. It’s not very flexible, of course, but at least it works and it does not require us to fetch-restrict anything.

It’s hard for me to tell how many more plugins for iscan are out there, but if your scanner is among them, feel free to send an ebuild for it on our Bugzilla and CC me — you can base yourself off media-gfx/iscan-plugin-perfection-v370 which is the latest I committed.

Unpaper (un)planning

So after just shy of an year of me forking unpaper to save it from the possible shutdown of BerliOS, and to update it to fit better in a 2012 distribution, I guess it’s time to have some ideas on how to move forward.

Now, while I haven’t done as much work on it as I’d have liked to begin with, we have a new series of packages, version 0.4.x, which has been packaged in Gentoo since day one (obviously), and is now available in Debian as well; these versions have a proper, Autotools-based build system, source code split in multiple files and cleaned up a little bit, and have a rewritten option parsing, although an imperfect one still.

The main issue with the option parsing is that the original parameters are not compatible with your average Unix long options — the end result is that I had to come up with enough hacks to have them parsed properly, and I can’t get the command to behave like a Unix command unless I also break compatibility with its original parameters. Which is something that might very well happen.

But there is another issue in all of this: the code still uses an handmade parser of netpbm-style image files, which is quite nasty to be honest, and likely prone to issues (I haven’t tried to see if I could overflow it, but I wouldn’t be surprised if it was possible; error handling also needs lots of work). It’s obvious that what I would like to do is replace the whole image loading and saving to use an external library of some kind, which also means that it would be possible to support more input and output formats at the same time — which is an especially good thing because right now during the testing phase I have to convert some PNGs to PNM to process.

My first try has been using gdk-pixbuf: the reason is that it’s already a commonly-used library, it has been split off Gtk lately, and using it and the rest of Glib it means that unpaper then has very little of its own utility functions, as Glib provides most of it, file access, option parsing and most importantly threadpools, which I wanted to use to be able to implement --jobs inside unpaper itself. Unfortunately the gdk-pixbuf library doesn’t seem to be extremely well suited for image processing as much as it is for decoding; saving files is clumsy and even the in-memory representation doesn’t feel extremely easy to play with.

The other solution I had in mind was to use libav for loading and saving — for sure its decoders and encoders are as optimised as it makes sense and will keep so, and it’s not so difficult to foresee that if the code is rewritten to be compatible with libav itself it could just become a piece of libavfilter, making unpaper just a frontend. The problem is that this still requires a lot of work, I’m pretty sure.

And there is one more issue here to say something about: for a while I won’t have as much motivation to work on unpaper for the simple reason that I won’t have a scanner! Since I couldn’t get an US Visa at this time, I won’t be able to actually move to the United States anytime soon, which put a hold to all my plans to ship or sell my stuff here and then buy what remains there. For this reason I won’t have as much use for unpaper as I had up to now. I don’t doubt at some point I’ll have again a scanner and the need to archive data (actually I’m quite sure this will happen sooner rather than later), so I will keep looking into improving unpaper in the mean time still, but it might take a backseat to other things.

Scanning documents with the GT-S50

You might remember that a couple of months ago I bought a high-end scanner for my ongoing mission to make my office as paperless as humanly possible to me.

In due time, I also replaced my previous scan2pdf sh script with something a bit more reliable, written in Ruby. The original reason was that the options supported by the four scanners I have worked with up to now – an Epson Perfection 2480/2580, a HP OfficeJet MFP with a “fake feeder”, a HP LaserJet network MFP, and now the GT-S50 – varied considerably, and I needed something that would actually allow me to choose the options to pass depending on the device itself. The end result is a Ruby script scan2pdf which is still not implementing all I wished for, but it comes much nearer than what I used before.

One of the most recent features came from a requirement I found when trying to get rid of some legal paperwork from some years ago: for whatever reason, in Italy, a number of legally-binding papers seem to still be printed and set in folded sheets (what is usually called foglio protocollo in Italian); this means that you got a single A3-sized sheet, folded in the middle to provide four A4-sized sides to write on. And actually cutting this down to reduce it to two A4 sheets is not an option. How do you scan that?

With a standard flatbed scanner it is relatively easy to scan it: you just scan the four sides one by one. On a sheet-fed scanner like the GT-S50, you can’t do so unless you use the plastic envelope that they give you, which allows to scan irregularly-shaped sheets. And even doing so, it is a bit of a problem because you either have to run four scans, or you have to interleave the external and internal sides. So the end result was implementing a --folded switch to the script, and there you go.

What I’m missing now is some kind of “presets” system, so that I don’t have to repeat the same options each time when I’m doing common scans: a folded scan requires me to use the envelope, which in turn means that the first 20mm of the scan need to be discarded (as it’s the envelope’s detection zone). And all the scans from the GT-S50 (as well as the Perfection 2580) are to be done in the middle-area, rather than starting from the far left or right (as it was done, instead, on the HP MFPs.

For this one, I’m actually open to suggestions: do anybody know a decent Ruby library to handle ini-style configuration files? I’m not keen on using YAML here, not only because I can’t stand the format, but also because it seems more natural to keep using the key/value pair files when there is no need for anything particularly complicated (multiple sections are welcome for presets themselves).

Gentoo and EPSON scanners

“Harry,” Susan said. “Have you ever heard of the paperless office?”

“Yeah,” I said. “It’s like Bigfoot. Someone says he knows someone who saw him, but you don’t ever actually see him yourself.”

Jim Butcher — Changes

Because of the way Italian bureaucracy is designed, most of the business communications I deal with are still in paper form. And since my business lately has been covering a number of different customers, with different degrees of formality, I’ve been pretty much swamped by paper. I have been looking for ways to reduce the amount of paper I have around, or at least the amount of paper I have to look at often enough, and the solution was that to get a scanner with auto-document feeder (ADF) and keep all the new paper document scanned before either shredding or archiving them.

I settled first for an HP OfficeJet multi-function printer (which jammed so many times it wasn’t funny, but at least was cheap) and then for a LaserJet one that I was given by a family friend who couldn’t use it via USB any longer. Unfortunately it was a M1522nf, which turns out has a defective formatter card which requires either baking or replacement. HP wouldn’t replace it without paying a couple hundreds of euro, which wouldn’t be worth it at all.

I went to look for a solution, with the understanding that my current laser printer (a Kyocera-Mita FS-1020D) is quite a charm to use, especially now that it is no longer connected to an Apple Airport Extreme, but rather to my Gentoo-based router whose cups instance is directly referenced by both Yamato and Raven so that I don’t have to run it on them any longer. I went to look at more professional, office-oriented scanners, since the only one I could find in “entry level” by HP listed a duty cycle of 100 scans/day (which I’m sure I would bust easily; a bank-related contract is usually longer).

The requirements, beside being able to scan a decent amount of sheets (more than 20) were generally the usual you’d expect for a person like me: it has to work with Linux, possibly 64-bit, with preferably no prebuilt, proprietary software. To be honest, this doesn’t seem to be feasible at all. And before somebody asks, no HP is not much better. Sure HPLIP is open source and Free, but to use the scanner in the aforementioned M1522nf, you have to install their proprietary plugin bits. Canon scanners, which seems to be the cheapest with Linux support for ADF, are supported by their driver, but it’s totally proprietary and more importantly it only works on 32-bit systems.

EPSON, instead, seems to have a more interesting approach. The Avasys-developed epkowa backend for sane is actually quite nice: it encapsulates a decently-sized open-source backend with a number of proprietary plugins (and in some cases firmware files). I already had a bit of experience with that backend before, since my flatbed scanner is a Perfection 24802580, but with that I couldn’t make use of it in a long time simply because the non-basic functionality (film scanning) is supported by a 32-bit only plugin.

After snooping around and poking my hardware supplier about it, I decided to get a GT-S50 (sheet-fed) document scanner, which is shaped more like an inkjet printer or a fax machine than a scanner. This time as well a plugin is needed, so based on the GT-F720 ebuild that was already in tree I wrote one for the required plugin (and also one for the GT-F500 plugin that is used by the Perfection above — but the only task I can use it for is to install the firmware file used by snapscan, so if you want to try it on 32-bit please let me know if it works at all!). You can now find iscan-plugin-gt-s80 (it includes support for both S50 and S80) and iscan-plugin-gt-f500 in tree already.

I got the scanner on Friday evening and set it up; the driver worked at the first try, and the iscan tool provided by Avasys/Epson works just right. On the other hand, my customized script over scanimage (as provided by sane-backends) was not working properly when scanning duplex (i.e. both sides of the sheet at once). In such a configuration, a single pass of the sheet in the ADF causes two pages to be read; the driver (or the firmware, not sure) caches the back side as the second page, and the next time sane is asked for a page, it returns the one already scanned. When doing the second scan, it was random whether it worked or not: a race condition.

I haven’t spent enough time on it yet to know where the problem lies exactly: it might be a bug in the scanimage frontend (given that iscan works fine), or a bug in the drivers. At any rate for now I worked it around by adding a sleep(1) before calling sane_start() in the file. You can find a hacked 1.0.22 ebuild for sane-backends in my overlay if you happen to have similar problems.

At the end of the day, I’m pretty happy with the device, it scans very nicely, very fast and has a high enough duty cycle for the kind of stress I could make it go through. Once I’ll be able to resolve the above-noted issue with scanimage it’ll work much better. But there are a few issues that still need to be solved, such as:

  • iscan needs to be set to use /var/lib/iscan rather than the current /var/lib/lib/iscan, and that requires plugins to be re-registered;
  • I need to find a way to register the plugin properly when installing in a different ROOT, which right now neither of my ebuilds do;
  • the iscan-plugin-gt-f720 ebuild needs to be brought up to speed with the other two, as right now it seems sub-par (among other things it installs the plugins in /usr — which, albeit being what AVASYS does on RedHat systems, is wrong, as they should go in /opt).

I dream the paperless office

And I know pretty well it’s something almost impossible to have; yet I’d like to have it because I’m succumbing in a ocean of paper right now. And paperwork as well.

While I have the Sony Reader to avoid having to deal with tons of dead tree books (although I do have quite a bit still, lots of which are still being consulted), I didn’t try before to clean up my archive of receipts, packaging slips, and stuff like that.

Time has come now since I have to keep some fuller, cleaner archive of invoices sent and received for my new activity as a self-employed “consultant”; I decided to scan and archive away (in a plastic box in my garage, that is) the whole of the job papers I had from before, as well as all my medical records, and the remaining parts of the archive. The idea was that by starting anew I could actually start keeping some time of accountability of what I receive, and spend, both for job and for pleasure. Together with the fact that is less stuff to bring around with me, this makes two things that would get me nearer toward actually moving out of home.

Sunday and Monday I spent about eight hours a day scanning and organising documents, trashing all the stuff I’m not interested in keeping an original of (that is stuff I’m glad I can archive, but that even if I lost is not that important), and putting away in the plastic box the important stuff (job and medical records, receipts for stuff that is already in warranty, etc.). I think I got through around 400 pages, on a flatbed scanner, without document feeder, assigning a name to each, and switching scans between 150 and 300 dpi, colour, grayscale and lineart scans.

I guess I’ll try to keep my archive more updated from now on by scanning everything as it arrives instead of waiting for it to pile up for twelve years (yes I got some receipts dating back to twelve years ago, like my first computer, a Pentium 133) and then trying to crunch it away in a few days. My wrist is aching like it never did before, for the sheer amount of sheets put on and removed from the scanner (I sincerely hope it’s not going to give up on me, it would be a bad thing).

Now I’m looking for a way to archive this stuff in a quick and searchable way, file-based structures don’t work that well, tagging the stuff would work better, but I have no idea what to use for that. If anybody has a free software based solution for archiving, that can be queried by the network too is a bonus, that it works on Mac OS X with Spotlight is a huge bonus, I’d be glad to hear it.

I’m also going to try out some software for accountability; I’ve heard good words of gnucash but never tried it before so I’m merging it right now; for now I don’t have enough invoices to send out that would give me reason to start writing my own software, but if there is something out there customisable enough I’d be glad to bite the bullet and get to use it. Spending my free time to work on software I need to work is not my ideal way to solve the problem.

Up to now I only worked very low profile, without having to invoice or keep records; luckily I have an accountant that can tell me what to do, but there are personal matters, including personal debts, credit cards and other expenses I finally want to take a good look at, so that I can extinguish them as soon as possible, and then start putting some away to pay for a car and a places to move to. Not easy to do I guess, but that’s what I hope to be able to do.

Frontends to command-line or libraries?

I know I’m still convalescing so I should be resting, not thinking about development problems, but this is something that ended up in my mind because I have one thing to absolutely do (scan the release documents from the hospital to send to my GP), and for which I miss an easy interface I could instruct my mother, or my sister, to use.

Don’t get me wrong, I know there are a few SANE frontends, starting from xsane itself, but the problem is that I don’t usually stop at scanimage when I do it from the console. What I usually do is to launch a batch scan to TIFF format, then use tiffcp to join the different TIFF files in a single file with multiple pages, and then use tiff2pdf to convert it to a PDF file, which is opened by any computer I might need to send the data to (and it also is quite smaller than the original TIFF file). Lately, I’ve started trying to add to the chain also unpaper, a tool that removes some of the defects usually found in scanning pages from books (like black borders and similar), which works on PNM (thus requiring a change in the scanning command, and a further conversion later on).

But I don’t want to start fleshing down how to actually write such a tool, or I might actually start writing it right now, which is not what I’m supposed to do while I’m convalescing.

What I want to think about is that here comes one huge debate between writing frontends for command-line tools, which just interfaces with them as process calling, or writing a full-fledged program that interfaces with the low-level libraries to provide similar functionalities.

I already happen to discuss that quite often since xine and mplayer embody the two spirits: xine has its library, frontends interface with that, and a single process is used; mplayer instead has a command interface, and frontends execute a new process for playing videos.

There are of course advantages and disadvantages, one easy to spot disadvantage to xine’s approach is that a crash or freeze in xine results in a crash or freeze of the frontend, which is something Amarok users have been unfortunately familiar with.

In the case of the scanning toolchain, though, I guess it’d be probably easier to use a frontend for the tools, as re-implementing all the different functionalities would be a non-trivial work.

The disadvantages of doing it this way, though, is that you’ll have to make sure that the tools don’t change their parameters between versions, otherwise it’d be a problem to ensure the correct functionality of the tool. Also, the tools need to provide enough options to control with granularity the execution of their task, which is something unpaper actually does, but in turn makes them almost unusable for final users.

I know I’m not going much anywhere with this post, I’m afraid, but I just wanted to reflect on the fact that to have a command line tool designed to be used by frontends, you almost certainly make its syntax so complex that users would fail to grasp the basic concepts, and in turn, you’d need a command line interface to the tool too… which is why there are so many scripts interfacing to ffmpeg for converting videos, I guess.

On the other hand, one can easily write such a frontend using scripting languages, even if it’s graphical, such as with ruby-gtk2 (think RubyRipper).

You know that your health took a bad turn when…

… your medical records folder is ten times thicker than the job contracts folder. I was cleaning up through the paperwork yesterday and today, and there are so many things.. Luckily I have most of my CT scans in digital format, beside the last one I did at the ER last week, and a cerebral CT from a few months ago. But the release paperwork I had only in printed form, which I needed to scan and “PDFize” on Tuesday to send to my GP — nice to have one who’s reachable via e-mail.

It’s the scanning that actually made me think a bit. I have the scanner a bit far from my workstation; to scan a five pages document I have to prepare the scanimage command in batch mode on the workstation, then walk all around my desk to get to the scanner, and then get my arm around the monitor to press the return key on the keyboard to start the scan.

The annoying thing is that the scanner has four buttons on it, that should be made just for the task of starting the scan. Unfortunately these buttons don’t work out of the box on Linux at all. There is a package, called scanbuttond that polls for them through libusb and then execute a custom script when they are pressed. But as you can guess, polling means it uses a lot of CPU, and the fact that they run a generic script makes it less integrable in a desktop environment. Of course it would be easy to port scanbuttond to submit the read buttons back into the kerel input subsystem so that they appear as generic events, but… I think this should be a task well suited for a kernel module, hooking them up directly in the input subsystem, so that evdev could pick them up and then a program could just wait for them as shortcuts to have some action.

I tried looking into writing something before, but I ended up stuck in a problem: would a kernel module interfacing with the scanner interfere with libusb access by sane itself? Last time I enquired Greg KH, he asked me to proivde lsusb -vv output but, sorry Greg, work piled up and I forgot about all of it (until yesterday when I had to scan some more documents). Well, if anybody wants to take a look, this is it for my current scanner:

Bus 001 Device 003: ID 04b8:0121 Seiko Epson Corp. Perfection 2480 Photo
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x04b8 Seiko Epson Corp.
  idProduct          0x0121 Perfection 2480 Photo
  bcdDevice            0.00
  iManufacturer           1 EPSON
  iProduct                2 EPSON Scanner
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           39
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           3
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered

Can anybody tell if it would be possible for a kernel module and sane to access it at once? :)

If there is the chance, I might look at it once I feel better. I haven’t written or touched a kernel module in so much time that I’m feeling like doing some work in that regard. And the code to actually get the data out of the scanner should be present in scanbuttond already, it’s just a matter of getting it “pushed in” rather than “polled for”.

Once I could get the buttons working, I’d probably be working on a GTK-based frontend for scanimage with handling of those, so I could just use those rather than having to set it up manually. Although the nice thing of scanimage is that, through that, tiffcp and tiff2pdf, I can quickly create a multi-page PDF of the scan (the first command creates a multi-page TIFF file, the second converts it to PDF), and if I do the scan in lineart (the perfect solution for B/W text) it also is tremendously small in size. I should try to have the same result with my frontend. My idea would be a light frontend written in Ruby, calling the commands directly.

Oh well, at any rate, this will have to wait till I’m feeling really better, or maybe it’s just unfeasible because of the way it’d need to access the scanner from kernel and libusb.

I should retire…

… and I mean in a small house in a small town in the alps, far away from computers, ADSL connections, and all these things.

Why? Well, if you follow my blog you know I had some health problems this summer, and the main suggestion of the doctors was trying to relax more. As it was, I really didn’t relax at all, Gentoo and xine and other Free Software projects sucking up all my time.

Now, today I was supposed to have a job interview, but in the last couple of days my blood pressure was a bit too high, and my doctor told me to just relax for some days, this included cancelling the meeting. Of course I need to keep controlled the pressure because I also have the warfarin to take, and I need to get my INR tested again, too.

So okay, after passing the morning reading, and finding out that this just make me think too many times about my problems, I decided to try my luck today by trying to learn how to write a Linux USB driver. Well, by trying again at least.

Last time I tried it was to see if I was able to implement a driver for a USB to serial adapter that I bought on eBay: out of four of them, three were PL2303 (perfectly working) and one used a W.ch chipset (a Chinese producer) that is not supported by Linux, and I failed to find any specification for it. I tried more than once to contact the producer, with no results. Sniffing the interaction between Windows and the device I was able just to find some commands. but nothing useful to actually make the device usable.

This time, I started thinking about the four buttons on my Epson scanner. The only thing I found to actually make them useful under Linux was the scanbuttond project, but I don’t really like its approach. The first problem is that the daemon polls for the status of the buttons, and this isn’t really a nice approach for the CPU. The second is that it executes commands, which aren’t really well put into context, and especially you can’t use it to provide commands to a scanning application.

The Linux kernel provides a very good way to handle buttons of any kind, the event devices. Even the ACPI buttons are now event devices, and those are easy to take into context: just make sure the application opens the correct event device, and that should take care of reading the buttons.

So I said to myself: “why can’t I try to write a driver for those buttons that shows the scanner as an event device?”. This time there was nothing to reverse, the code was written already; in userspace using libusb, but it’s there as a reference. And I started.

I almost forgot having a copy of Linux Device Drivers, it arrived not long before I ended up in the hospital, and I tried to remove most of the last weeks before that, as well as the first weeks in the hospital. I’d like to remove the whole hospital experience, but that’s near to impossible.

So I started reading the USB part, and I was able to come to something that at least loaded and identified the scanner. I did make some mistake on the disconnection function, as my kernel paniced; I restarted with the intention of using VMware to test it next time, at least then I panic just the guest OS and I can restart it without stopping my work.

But when I restarted I also read the reply to the mail I sent to Daniel (dsd) before, and he gave me the bad news: if the interface I need to read the buttons data from is the same that SANE needs to access the scanner, then it won’t work anymore. And of course, Epson didn’t use different interfaces, it’s just the same. Different endpoint, but still same interface.

So I’ll probably just rm -rf the code I wrote up to now, as it makes it useless… it can turn a scanner into a 4-buttons keyboard, but I doubt this is what users want.

Seems like this is just another failure for me…

Update from the Java front

So, yesterday I blogged about Free Java implementation Kaffe, and today I have some updates on that.

First of all, dalibor on #kaffe suggested me to try CVS head, and of course it make sense, considering the fluent state of the project, so I added a CVS live ebuild to my overlay, keyworded -*.

The he explained that the problem is with GCC 4.1 that changed something in exception handling, and provided me a patch to try out, that actually worked. The main problem is that it forces -O0 to build.

After patching Kaffe, and building it with those totally un-optimised flags, the result is quite good, as Yahoo games works fine, albeit quite slow, and the CPU usage is about 70%. This makes Kaffe un-optimal for people relying a lot on Java, but make it good enough for me that I don’t use it much a part from playing literati from time to time. Just a note: you need to disable KIO support in Konqueror or loading java applets will fail.

Following this Java improvement, I have to thank sbriesen for helping me setting up my Epson scanner; it seems like I need only the firmwares out of media-gfx/iscan, and then snapscan backend works fine. Yuhuu.

Okay now I can actually reduce my OSX usage to the minimum, as to do copies I can just use Kooka. Unfortunately I wasn’t able to find an application (yet) that is able to read the negatives out of the scanner and split them in three different photos as it’s done by Epson’s utilities. I’m not a graphic person, so I have no idea where to start, and thus I can’t help with it, but I’m ready to test anything if someone is going to work on such an application.

So now the other main binary thing I rely on is the nvidia driver, that’s going to take the way of the sea next week most likely, when my 9250 will come. €55 might e a good price for Freedom. I’ll tell if I have any quirk with that card that might be needed.

Oh yes and I (with a bit of luck rarely) rely on win32codecs for some wmv9 files, but that’s not going to be a problem as soon as FFmpeg will support VC-1 codec, so I’m not even considering that an issue, for how little times I actually need it.