unpaper: re-designing an interface

This is going to be part of a series of post that will appear over the next few months with plans, but likely no progress, to move unpaper forward. I have picked up unpaper many years ago, and I’ve ran with it for a while, but beside a general “code health” pass over it, and back and forth on how to parse images, I’ve not managed to move the project forward significantly at all. And in the spirit of what I wrote earlier, I would like to see someone else pick up the project. It’s the reason why I create an organization on GitHub to keep the repository in.

For those who are not aware, unpaper is a project I did not start — it was originally written by Jens Gulden, who I understand worked on its algorithms as part of university. It’s a tool that processes scanned images of text pages, to make them easier to OCR them, and it’s often used as a “processing step” for document processing tools, including my own.

While the tool works… acceptably well… it does have a number of issues that always made me feel fairly uneasy. For instance, the command line flags are far from standard, and can’t be implemented with a parser library, relying instead on a lot of custom parsing, and including a very long and complicated man page.

There’s also been a few requests of moving the implementation to a shared library that could be used directly, but I don’t feel like it’s worth the hassle, because the current implementation is not really thread-safe, and that would be a significant rework to make it so.

So I have been having a bit of a thought out about it. The first problem is that re-designing the command line interface would mean breaking all of the programmatic users, so it’s not an easy decision to take Then there’s been something else that I learnt about that made me realize I think I know how to solve this, although it’s not going to be easy.

If you’ve been working exclusively on Linux and Unix-like systems, and still shy away from learning about what Microsoft is doing (which, to me, is a mistake), you might have missed PowerShell and its structured objects. To over-simplify, PowerShell piping doesn’t just pipe text from one command to another, but structured objects that are kept structured in and out.

While PowerShell is available for Linux nowadays, I do not think that tying unpaper to it is a sensible option, so I’m not even suggesting that. But I also found out that the ip command (from iproute2) has recently received a -J option, which, instead of printing the usual complex mix of parsable and barely human readable output, generates a JSON document with the same information. This makes it much easier to extract the information you need, particularly with a tool like jq available, that allows “massaging” the data on the command line easily. I have actually used this “trick” at work recently. It’s a very similar idea to RPC, but with a discrete binary.

So with this knowledge in my head, I have a fairly clear idea of what I would like to have as an interface for a future unpaper.

First of all, it should be two separate command line tools — they may both be written in C, or the first one might be written in Python or any other language. The job of this language-flexible tool is to be the new unpaper command line executable. It should accept exactly the same command line interface of the current binary, but implement none of the algorithm or transformation logic.

The other tool should be written in C, because it should just contain all the current processing code. But instead of having to do complex parsing of the command line interface, it should instead read on the standard input a JSON document providing all of the parameters for the “job”.

Similarly, there’s some change needed to the output of the programs. Some of the information, particularly debugging, that is currently printed on the stderr stream should stay exactly where it is, but all of the standard output, well, I think it makes significantly more sense to have another JSON document from the processing tool, and convert that to human-readable form in the interface.

Now, with a proper documentation of the JSON schema, it means that the software using unpaper as a processing step can just build their job document, and skip the “human” interface. It would even make it much easier to write extensions in Python, Ruby, and any other language, as it would allow exposing a job configuration generator following the language’s own style.

Someone might wonder why I’m talking about JSON in particular — there’s dozens of different structured data formats that could be used, including protocol buffers. As I said a number of months ago, the important part in a data format is its schema, so the actual format wouldn’t be much of a choice. But on the other hand, JSON is a very flexible format that has good implementations in most languages, including C (which is important, since the unpaper algorithms are implemented in C, and – short of moving to Rust – I don’t want to change language).

But there’s something even more important than the language support, which I already noted above: jq. This is an astounding work of engineering, making it so much easier to handle JSON documents, particularly inline between programs. And that is the killer reason to use JSON for me. Because that gives even more flexibility to an interface that, for the longest time, felt too rigid to me.

So if you’re interested to contribute to an open source project, with no particular timeline pressure, and you’re comfortable with writing C — please reach out, whether it is to ask questions for clarification, or with a pull request to implement this idea altogether.

And don’t forget, there’s still the Meson conversion project which I also discussed previously. For that one, some of the tasks are not even C projects! It needs someone to take the time to rewrite the man page in Sphinx, and also someone to rewrite the testing setup to be driven by Python, rather than the current mess of Automake and custom shell scripts.

The Importance of Project Boxes

Given my background, you may think I’m referring to servers — after all I ran the tinderbox for a long while. But no, in this case I’m referring to actual, physical boxes. And a few virtual ones as well.

See, when I listed my electronics projects, it probably sounded like my house is now top-to-bottom filled with random electronics stuff. This might have been the case if I was doing that in Dublin, but nowadays I’m not alone, and while my wife accepts the need for my random projects, making a total mess of the apartment is probably not a great idea. Indeed the “study” (that is, the second bedroom turned into a home office) is not quite sorted out, but it’s also not a total mess right now (you can see it in the test Live whiteboarding).

The way I’m trying to keep all of this together, is through a significant number of boxes. I already bought a number of LEGO boxes back in Dublin — the reason for the choice was that those boxes are particularly designed to be used by kids, and that made them particularly sturdy. Unfortunately they are also not particularly big. That made them not particularly useful for storing more bulky items, such as my multimeter, the dremel or most of my screwdrivers’ set. I did bring a toolbox with me from Italy – which includes not one but two hammers, more screwdrivers, one older soldering iron, and other similar tools – but this ended up being the kind of toolbox where tools end up to die, because it’s just so uncomfortable to use.

Instead I now bought bigger, bulkier boxes — using one to store the tools, and another one to store my “electronics” stuff, most of which is stored in tiny boxes inside — so that I have a box of transistors, two of resistors, a couple of Dupont connectors, and so on. This makes it easier to sort through what I need at any one time. I considered using bigger boxes with more compartments, but it turns out that most of the time I only need three or four smaller boxes, and sorting between smaller boxes is easier. I have unfortunately not found a very good box to hold the soldering kit – TS-100 and so on – but I might end up looking for something with foam inserts if I ever figure out how to properly design those, after all.

The LEGO boxes are now (mostly) project boxes: one of them has all of the glucometers that I’ve either already reversed or plan to reversing soon; another one has the R205 boards that I never made much progress on, and so on. The fact that I have so many half-complete projects is indeed disheartening when I think of it. Unfortunately there’s reason for that: either lack of skills on my part, the lack of time for myself, or the lack of tooling at the right time. I can for instance tell you that I definitely would have a much easier time nowadays trying to solder those those pins, not just because I improved my soldering skills, but also because I now have tools that I didn’t even know existed back then, including the Dupont crimper.

And this is where the project boxes come into play. Being able to put everything for one particular project into a box and not worry about it for a while is a liberating experience. I had to literally do that when I started my new job — it’s not that difficult passing from one Big Tech company to another, but it is time consuming, and after spending nearly eight straight hours in front of a monitor watching videos and/or reading tutorials, the last thing I wanted was to spend some more time in front of the computer… having all the half-projects in front of me was tough, and I preferred shoving everything into a project box, and going back to it a week or so later — I have indeed gone back to one of my projects already and will likely have a blog post about it later.

But as I said, there’s also “metaphorical” project boxes — that work for me both at work and at home: there are times when working on the same project for months at a time is not feasible. It may be that what was once a priority isn’t anymore, or a higher priority comes out. Or maybe you’re just too burnt out to work on one thing, and decided that it’s time to work on something else for a while. Having the ability to set aside a project, and come back to it later, having state and notes about what something was about is something important. For most of my opensource projects, I leave my own notes in the blog, which is why there’s so many posts of old projects that have not gone anywhere. For work projects, I try to leave design docs or notes, and for other stuff, I started having proper journals dedicated to the ideas. That’s how Birch Books started as well.

The main advantage I found for doing this at work is that it allows collecting rakes without losing track of the main objective of your job. Many of the examples I gave in the previous blog post on the topic have been months-long efforts. Some of those took two years to close the reported issues! And in at least one case, the work had to literally be halted, and wait for two quarters, because one of the blockers could only be solved by deleting the whole source of a service that was being replaced.

So what I’m advocating about is the ability to keep things around, but out of sight. It’s the reason why I can’t stand certain policies in projects (at work and FLOSS alike) that pretty much disallow “long-standing bugs”: sometimes you want to keep all the information you have on a project together, and a bug/issue/task that has been opened for a number of years might just be the right way — closing it for the sake of an OKR target is unlikely to do anyone a good service.

And also allowing ourselves (all of us), to box up projects — whether it is for a while or forever. Being reminded of the stuff you have not completed is not going to make you feel any better, in my experience. Being able to say “Yeah it’s in the box, one day I might or might not get to it” is a way to breathe in, and give yourself space.

Upcoming electronics projects (and posts)

Because of a strange alignment between my decision to leave Google to find a new challenge, and the pandemic causing a lockdown of most countries (including the UK, where I live), you might have noticed more activity on this blog. Indeed for the past two months I maintained an almost perfect record of three posts a week, up from the occasional post I have written in the past few years. In part this was achieved by sticking to a “programme schedule” — I started posted on Mondays about my art project – which then expanded into the insulin reminder – then on Thursday I had a rotating tech post, finishing the week up with sARTSurdays.

This week it’s a bit disruptive because while I do have topics to fill in the Monday schedule, they start being a bit more scatterbrained, so I want to give a bit of a regroup, and gauge what’s the interest around them in the first place. As a starting point, the topic for Mondays is likely going to stay electronics — to follow up from the 8051 usage on the Birch Books, and the Feather notification light.

As I have previously suggested on Twitter, I plan on controlling my Kodi HTPC with a vintage, late ’80s Sony SVHS remote control. Just for the craic, because I picked it up out of nostalgia, when I went to Weird Stuff a few years ago — I’m sad it’s closed now, but thankful to Mike for having brought me there the first time. The original intention was to figure out how the complicated VCR recording timer configuration worked — but not unexpectedly the LCD panel is not working right and that might not be feasible. I might have to do a bit more work and open it up, and that probably will be a blog post by itself.

Speaking of Sony, remotes and electronics — I’m also trying to get something else to work. I have a Sony TV connected to an HDMI switcher, and sometimes it get stuck with the ARC not initializing properly. Fixing it is relatively straightforward (just disable and re-enable the ARC) but it takes a few remote control button presses… so I’m actually trying to use an Adafruit Feather to transmit the right sequence of infrared commands as a macro to fix that. Which is why I started working on pysirc. There’s a bit more than that to be quite honest, as I would like to have a single-click selection of inputs with multiple switchers, but again that’s going to be a post by itself.

Then there’s some trimming work for the Birch Books art project. The PCBs are not here yet, so I have no idea if I have to respin them yet. If so, expects a mistakes-and-lessons post about it. I also will likely spend some more time figuring out how to make the board design more “proper” if possible. I also still want to sit down and see how I can get the same actuator board to work with the Feather M0 — because I’ll be honest and say that CircuitPython is much more enjoyable to work with than nearly-C as received by SDCC.

Also, while the actuator board supports it, I have currently left off turning on the fireplace lights for Birch Books. I’m of two minds about this — I know there are some flame effect single-LEDs out there, but they don’t appear to be easy to procure. Both bigclive and Adam Savage have shown flame-effect LED bulbs but they don’t really work in the small scale.

There are cheap fake-candle LED lamps out there – I saw them the first time in Italy at the one local pub that I enjoy going to (they serve so many varieties of tea!), and I actually have a few of them at home – but how they work is by using PWM on a normal LED (usually a warm light one). So what I’m planning on doing is diving into how those candles do that, and see if I can replicate the same feat on either the 8051 or the Feather.

I don’t know when the ESP32 boards I ordered will arrive, but probably will spend some time playing with those and talking about it then. It would be nice to have an easy way to “swap out the brains” of my various projects, and compare how to do things between them.

And I’m sure that, given the direction this is going, I’ll have enough stuff to keep myself entertained outside of work for the remaining of the lockdown.

Oh, before I forget — turns out that I’m now hanging out on Discord. Adafruit has a server, which seems to be a very easygoing and welcoming way to interact with the CircuitPython development team, as well as discussing options and showing off. If you happen to know of welcoming and interesting Discord servers I might be interested in, feel free to let me know.

I have not forgotten about the various glucometers I acquired in the past few months and that I still have not reversed. There will be more posts about glucometers, but for those I’m using the Thursday slot, as I have not once gone down to physically tapping into them yet. So unless my other electronics projects starve out that’s going to continue that way.

The future of Unpaper

You might have read it already, or maybe you haven’t, but it looks like Berlios is going to close down at the end of the year. I wouldn’t go as far as calling it the end of an era, but we’re pretty close. Even I have a few projects that are (still) hosted on Berlios and I need to migrate.

Interestingly, Berlios is also the original hosting for unpaper which makes me forking it a couple of months ago a very good move. Especially since if I waited too long, the repository wouldn’t have been available.. even if it’s true that the repository didn’t really contain anything useful, as it was just an import of the 0.3 sources.

At any rate, since Jens didn’t reply to my inquiries, I’ve decided to start working on a more proper takeover of the project. I have created an unpaper project page on my website, to which I switched the live ebuild’s HOMEPAGE and GitHub’s website, as well as created an ohloh project to track the development.

Oh, and I released version 0.4. Yeah I guess this was the first thing to write about, but I wanted to make it less obvious.

The new release is basically simply the first cleanup I worked on, so new build system, no changes in parameters, man page and so on. Right after releasing 0.4 I merged the changes from the new contributor, Felix Janda, who not only took the time to break up the code in multiple source files, but also improved the blur filter.

Now, the next release is likely going to be 2.0; why skipping the 1.0 release? Well, it’s not really skipping. Before Berlios shuts down I wanted to copy down the previous list of downloads so I can mirror those as well, and what I found is that… the original version number series started with 1.0, so it’s mucked up badly; in Gentoo we have no problem since 0.3 was the only one we had, but for the sake of restoring consistence, the next version is going to be 2.0.

What is going to happen with that release? Well, for sure I want to rewrite the command-line options parsing. Right now it’s very custom, as long options are sometimes prefixed with one, sometimes with two dashes, and in general it doesn’t fit the usual unix command lines; not counting the fact that the parsing is done as you go with a series of strcmp() calls, which is not what one expects, usually. I intend to rewite this with getopt_long() but the problem with that is that it will break the command-line compatibility with unpaper 0.3, which is not something I’m happy about. But we’ve got to do that sooner or later if we want a more well-blent tool.

I hope to also be able to hook into the code a different way to load and save images, using some already present image decoding and encoding library, so that it can digest images in different formats than simple PNM. In particular, I’d like to be able to execute as a single pass the conversion from multiple files to a multi-page TIFF document, which requires quite a bit of work indeed. But one can dream, can’t I?

In the mean time, I hope to find some time this week to find a way to generate man pages on my server so that I can publish more complete documentation for both Ruby-Elf and unpaper itself. This is likely going to be difficult, since I’m starting some new tasks next week but.. you never know.

You know that your health took a bad turn when…

… your medical records folder is ten times thicker than the job contracts folder. I was cleaning up through the paperwork yesterday and today, and there are so many things.. Luckily I have most of my CT scans in digital format, beside the last one I did at the ER last week, and a cerebral CT from a few months ago. But the release paperwork I had only in printed form, which I needed to scan and “PDFize” on Tuesday to send to my GP — nice to have one who’s reachable via e-mail.

It’s the scanning that actually made me think a bit. I have the scanner a bit far from my workstation; to scan a five pages document I have to prepare the scanimage command in batch mode on the workstation, then walk all around my desk to get to the scanner, and then get my arm around the monitor to press the return key on the keyboard to start the scan.

The annoying thing is that the scanner has four buttons on it, that should be made just for the task of starting the scan. Unfortunately these buttons don’t work out of the box on Linux at all. There is a package, called scanbuttond that polls for them through libusb and then execute a custom script when they are pressed. But as you can guess, polling means it uses a lot of CPU, and the fact that they run a generic script makes it less integrable in a desktop environment. Of course it would be easy to port scanbuttond to submit the read buttons back into the kerel input subsystem so that they appear as generic events, but… I think this should be a task well suited for a kernel module, hooking them up directly in the input subsystem, so that evdev could pick them up and then a program could just wait for them as shortcuts to have some action.

I tried looking into writing something before, but I ended up stuck in a problem: would a kernel module interfacing with the scanner interfere with libusb access by sane itself? Last time I enquired Greg KH, he asked me to proivde lsusb -vv output but, sorry Greg, work piled up and I forgot about all of it (until yesterday when I had to scan some more documents). Well, if anybody wants to take a look, this is it for my current scanner:

Bus 001 Device 003: ID 04b8:0121 Seiko Epson Corp. Perfection 2480 Photo
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x04b8 Seiko Epson Corp.
  idProduct          0x0121 Perfection 2480 Photo
  bcdDevice            0.00
  iManufacturer           1 EPSON
  iProduct                2 EPSON Scanner
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           39
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           3
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered

Can anybody tell if it would be possible for a kernel module and sane to access it at once? :)

If there is the chance, I might look at it once I feel better. I haven’t written or touched a kernel module in so much time that I’m feeling like doing some work in that regard. And the code to actually get the data out of the scanner should be present in scanbuttond already, it’s just a matter of getting it “pushed in” rather than “polled for”.

Once I could get the buttons working, I’d probably be working on a GTK-based frontend for scanimage with handling of those, so I could just use those rather than having to set it up manually. Although the nice thing of scanimage is that, through that, tiffcp and tiff2pdf, I can quickly create a multi-page PDF of the scan (the first command creates a multi-page TIFF file, the second converts it to PDF), and if I do the scan in lineart (the perfect solution for B/W text) it also is tremendously small in size. I should try to have the same result with my frontend. My idea would be a light frontend written in Ruby, calling the commands directly.

Oh well, at any rate, this will have to wait till I’m feeling really better, or maybe it’s just unfeasible because of the way it’d need to access the scanner from kernel and libusb.

Summer of Code, Gentoo and other projects

So it seems we got accepted as organisation for Google Summer of Code 2008! And so were FFmpeg and FreeBSD (at least, I heard about those on a few blog).

I wish to remember the users who’re interested in Gentoo/FreeBSD that the main way to improve Gentoo/FreeBSD (once I find time to take back the project again, and make FreeBSD 7.0 ebuilds) is to improve FreeBSD itself. So if you want, you can apply for their SoC and still help the Gentoo project!

And FFmpeg is also an important project for me to have new people working on it: xine is based on that, so it’s a very important project. If you feel you can actually work on that area, join that SoC!

But of course make sure to check out Gentoo ideas, and feel free to contact me if you want further information on the project I proposed myself.

And if you still don’t know where to apply, check out the ideas for the rest of Summer of Code 2008!