Bloody upstream

Please note, this post is likely to be interpreted as a rant. From one point of view it is. It’s mostly a general rant geared toward those upstreams that is generally impossible to talk into helping us distribution out.

The first one is the IEEE — you might remember that back in April I was troubled by their refusal to apply a permissive license to their OUI database, and actually denied that they allow redistribution of said database. A few weeks ago I had to bite the bullet and added both the OUI and the IAB databases to the hwids package that we’re using in Gentoo, so that we can use them on different software packages, including bluez and udev.

While I’m trying not to bump the package as often as before, simply because the two new files increase the size of the package four times. But I am updating the repository more often so that I can see if something changes and could be useful to bump it sooner. And what I noticed is that the two files are managed very badly by IEEE.

At some point, while adding one entry to the OUI list, the charset of the file was screwed up, replacing the UTF-8 with mojibake then somebody fixed it, then somebody decided that using UTF-8 was too good for them and decided to go back to pure ASCII, doing some near-equivalent replacement – although whoever changed ß to b probably got to learn some German – then somebody decided to fix it up again … then again somebody broke it while adding an entry, another guy tried to go back to ASCII, and someone else fixed it up again.

How much noise is this in the history of the file? Lots. I really wish they actually wrote a decent app to manage those databases so they don’t break them every other time they have to add something to the list.

The other upstream is Blender. You probably remember I was complaining about their multi-level bundling ad the fact that there are missing license information for at least one of the bundled libraries. Well, we’re now having another problem. I was working on the bump to 2.65, but now either I return to bundle Bullet, or I have to patch it because they added new APIs to the library.

So right now we have in tree a package that:

  • we need to patch to be able to build against a modern version of libav;
  • we need to patch to make sure it doesn’t crash;
  • we need to patch to make it use over half a dozen system libraries that it otherwise bundles;
  • we need to patch to avoid it becoming a security nightmare for users by auto-executing scripts in downloaded files;
  • bundles libraries with unclear licensing terms;
  • has two build systems, with different features available, neither of which is really suitable for a distribution.

Honestly, I reached a point where I’m considering p.masking the package for removal and deal with those consequences rather than dealing with Blender. I know it has quite a few users especially in Gentoo, but if upstream is unwilling to work with us to make it fit properly, I’d like users to speak to them to see that they get their act together at this point. Debian is also suffering from issues related to the libav updates and stuff like that. Without even going into the license issues.

So if you have contacts with Blender developers, please ask them to actually start reducing the amount of bundled libraries, decide on which of the two build systems we should be using, and possibly start to clear up the licensing terms of the package as a whole (including the libraries!). Unfortunately, I’d expect them not to listen — until maybe distributions, as a whole, decide to drop Blender because of the same reasons, to make them question the sanity of their development model.

Who said that IDs wouldn’t have license issues?

When I posted about the hwids data I was not expecting to find what I just found today, and honestly, I’m wondering if I’m always looking for stuff under rocks.

My reason to split out the ID databases off their respective utilities (pciutils and usbutils) was simple enough: I didn’t need to deal with the two utilities, both of which are GPL-2, when the database themselves are available under the BSD 3-clauses license; it was just a matter of removing code, and avoiding auditing of projects that we don’t need to rely upon.

The fact that it was still a pet peeve of mine to not have an extra package taking care of it, rather than bundling them, was just an added bonus.

So after creating a silly placeholder which is fine for our needs here, with the blessing of Greg I created a new package, sys-apps/hwids (I wanted to call it hwdata, but we have both gentoo-hwdata and redhat-hwdata that install very different stuff), which has its own repository and with a live ebuild that simply fetches the files out of the respective website. I’m going to update the package weekly, if there are changes, so that we always have a more up-to-date version of it, and we won’t be needing the network cron scripts at all either.

I’ve also updated lshw to support the split package, so that it doesn’t install its own ids files anymore… of course that is only half the work done there, since the lshw package has two more datafiles: oui.txt and manuf.txt. The latter comes out of Wireshark, while the former is downloaded straight from IEEE’s Public OUI registry and that’s where the trouble starts.

The problem is that while you’re free to download the oui.txt file, you won’t find any kind of license on the file itself. I’ve sent a request to IEEE for some clarification on the matter and their answer is a clear “you cannot redistribute that file” (even though Ulrich, while not a lawyer, pointed out that it’s factual information which might not be copyrightable at all — see Threshold of originality for further details.

So why would I care about that single file given that lshw is a minor package by itself, and shouldn’t matter that much? Well, the answer is actually easy to give: bluez also contains a copy of it. And we’re redistributing that for sure, at least in source form. Sabayon is actually distributing binaries of it.

Interestingly enough, neither Debian’s lshw package nor their Bluez one do install the oui.txt file and I wouldn’t be surprised if their source archives have been neutered made Free by removing the distributed copy of the file.

What should we do about this? Unfortunately, that’s one question I don’t have an answer for myself yet, but I guess it should be clear enough that you can’t always assume that what upstream declares to be the case… actually is the case, especially for what concerns licensing. And this is the reason why, even though we don’t have any problem with releasing the source of all the GPL’d packages we have, we’d like to reduce as much as possible the amount of licenses I have to go through.