I’m told Rust is great, where are the graphics libraries?

While I’m still a bit sour that Mozilla decided to use the same name for their language as an old project of mine (which is not a new thing for Mozilla anyway, if someone remembers the days of Phoenix and Firebird), I have been looking from the sideline as the Rust language as a way forward to replace so many applications of embedded C, with a significantly safer alternative.

I have indeed been happy to see so much UEFI work happening in Rust, because it seems to me like we came far enough that we can sacrifice some of the extreme performance of C for some safety.

But one thing that I still have not seen is a good selection of graphics libraries, and that is something that I’m fairly disappointed by. Indeed, I have been told that there are Rust bindings for the classic C graphics libraries — which is pointless, as then the part that needs safety (the parsing) is still performed in C!

The reason why I’m angry about this is that I still have one project, unpaper, which I inherited as a big chunk of C and could definitely be rewritten into a safer language. But I would rather not do so in a higher level language like Python due to the already slow floating point calculations and huge memory usage.

Right now, unpaper is using libav, or ffmpeg, or something with their interface, depending on how much they fought this year. This is painful, but given that each graphic library implements interfaces in different ways, I couldn’t find a better and safe way to implement graphics processing. I was hoping that with all the focus on Rust out there, particularly from Mozilla, implementing graphics parsing libraries would be high in the list of priorities.

I think it’s librsvg that was ported to Rust — which was probably a great idea to prioritize, given it is exactly the type of format where C performs very poorly: string parsing. But I’m surprised nobody tried to make an API-compatible libpng or libtiff. It sounds to me like Rust is the perfect language for this type of work.

At any rate, if anyone finally decides to implement a generic graphic file input/output library, with at least support for TIFF, PNM and PNG, I’d love to know. And after that I would be happy to port unpaper to it — or if someone wants to take unpaper code as the basis to reimplement it as a proof of concept, that’d be awesome.

The problem for a lot of these libraries is that you have to maintain support for a long list of quirks and extensions that over time piled up on the formats. And while you can easily write tests to maintain bit-wise compatibility with the “original” C language based libraries for what concerns bitmap rendering (even for non-bitmap graphics such as JPEG and WebP), there are more things that are not obvious to implement, such as colour profiles, and metadata in general.

Actually, I think that there is a lot of space here to build up a standard set of libraries for graphics libraries and metadata, since there’s at least some overlapping between these, and having a bigger group of people working on separate, but API-similar libraries for various graphic formats would be a significant advantage for Rust over other languages.

unpaper and libav

I’ve resumed working on unpaper since I have been using it more than a couple of times lately and there has been a few things that I wanted to fix.

What I’ve been working on now is a way to read input files in more formats; I was really aggravated by the fact that unpaper implemented its own loading of a single set of file formats (the PPM “rawbits”); I went on to look into libraries that abstract access to image formats, but I couldn’t find one that would work for me. At the end I settled for libav even though it’s not exactly known for being an image processing library.

My reasons to choose libav was mostly found in the fact that, while it does not support all the formats I’d like to have supported in unpaper (PS and PDF come to mind), it does support the formats that it supports now (PNM and company), and I know the developers well enough that I can get bugs and features fixed or implemented as needed.

I have now a branch can read files by using libav. It’s a very naïve implementation of it though: it reads the image into an AVFrame structure and then convert that into unpaper’s own image structure. It does not even free up the AVFrame, mostly because I’d actually like to be able to use AVFrame instead of unpaper’s structure. Not only to avoid copying memory when it’s not required (libav has functions to do shallow-copy of frames and mark them as readable when needed), but also because the frames themselves already contain all the needed information. Furthermore, libav 12 is likely going to include libavscale (or so Luca promised!) so that the on-load conversion can also be offloaded to the library.

Even with the naïve implementation that I implemented in half an afternoon, unpaper not only supports the same input file as before, but also PNG (24-bit non-alpha colour files are loaded the same way as PPM, 1-bit black and white is inverted compared to PBM, while 8-bit grayscale is actually 16-bit with half of it defining the alpha channel) and very limited TIFF support (1-bit is the same as PNG; 8-bit is paletted so I have not implemented it yet, and as for colour, I found out that libav does not currently support JPEG-compressed TIFF – I’ll work on that if I can – but otherwise it is supported as it’s simply 24bpp RGB).

What also needs to be done is to write out the file using libav too. While I don’t plan to allow writing files in any random format with unpaper, I wouldn’t mind being able to output through libav. Right now the way this is implemented, the code does explicit conversion back or forth between black/white, grayscale and colour at save time, and this is nothing different than the same conversion that happens at load time, and should rather be part of libavscale when that exists.

Anyway, if you feel like helping with this project, the code is on GitHub and I’ll try to keep it updated soon.

Disk images

As I wrote before I’m currently converting my video archive to a format compatible with iTunes, AppleTV and PlayStation3. To do so, I’m archiving it directly on my OSX-compatible partitions, so that iTunes can access it. I’m actually thinking about moving it on an NFS-shared XFS partition, but for now it’s just HFS+.

To do so, though, I had to repartition the external 1TB hard disk, to give more space for the iTunes Library partition. For a series of reasons, I had to backup the content of the disk before, and now I’m restoring the data. To do the backup, I used the DMG image format that is native to OSX.

The nice thing of DMG is that it transparently handles encrypted images, compressed images, read-only images and read-write images. The compressed image format, that is what I used, was able to cut in half the space used up in the partition.

Now of course I know DMG is just a glorified raw image format, and that there are a lot of alternatives for Linux, but this made me think a bit about what the average user knows about disk and partition images.

I know lots of people who think that a raw image taken with dd(1) is good enough to be stored and used for every use. I don’t agree with that, but I can understand what the problem is with understanding why a raw image taken with dd is not just “good enough”.

The problem is that a lot of users ignore the fact that in a partition, together with the actual files’ data, there is space occupied by the filesystem metadata, the journal, and of course all the unused space is not always contiguous and it’s not always zeroed out.

Let’s take a common ad easy example that a lot of users who had to use Windows at one time will have no problem understanding: the FAT32 filesystem.

FAT filesystems tend to fragment a lot. Fragmentation does not only mean that files are sparse around the disk, but also that the free space is sparse between the fragments of files. Also, when you delete a file, its content is not removed from the disk, as you can guess if you ever used undelete-like tools to restore files that were deleted.

When you compress with tools like bzip2 or similar a file, the fact that it’s fragmented does not get in the way of the compression: if the file contains the same data repeated over and over it will be compressed quite easily. If the file is fragmented the same does not apply.

The fact that the free space is not zeroed out can cause lots of problems because a perfectly defragmented partition with 40GB of unused space cannot describe the unused space as “40GB of 0s”… unless.

Unless the compression algorithm knows about the filesystem. If, when you take an image of the partition, you use a tool that knows about the format of the partition, it starts to be much more useful. For FAT, that would mean that a tool could just move all the files’ data at the start of the partition, put all the unused space at the end, and consider it empty, zeroed out. The result is no more a 1:1 copy, but it’s probably good enough.

Now of course an alternative is just to use an archiving tool that can actually get all the files’ metadata (attributes, permissions, and so on), then you don’t have to worry about unused space at all. But that assumes you can use custom tools both for creating the image and to restore it. Creating an image of a partition could be quite easy to do with an already set-up complex tool, but there might be the need for the restore part of the code to be as lightweight as possible.

Now, I’m no expert on filesystems myself, so I might be wrong, or there might be the software doing this already out there. I don’t know. But I think it wouldn’t be too far fetched to think that there might be a software capable of taking an hard drive with an MBR partition table, with two FAT32 filesystems on it, both fragmented, with the unused space not zeroed out at all, and creating an image file that only needs bzip2 to be uncompressed, and dd to be written, but still providing a much smaller image file than a simple raw image compressed with bzip2, or maybe even rzip.

Such a tool could easily find duplicated files (which happens a lot on Windows because of duplicated DLLs, but can easily happen on Linux too because of backup copies for instance), and put them one near the other so to improve compression.

I know this post is quite vague on details, and that’s because as I said I’m not much of an expert on the field. I just wanted to make some users reflect on the fact that a simple raw image is just not always the perfect solution if what you need is efficiency in storage space.