Why I Care About Licensing

Both on the blog and on Twitter, I have ranted at length at projects missing licensing information altogether, or not providing licensing information on specific files, or providing conflicting licensing information. As you can imagine, this is a topic that I’m very attached to, which is why I have been following REUSE guidelines to make sure that all my (currently active) projects follow the specification.

Unfortunately this care is not shared with many developers, even those who consider themselves part of the Free Software movement, and this causes friction, poisons the well in both directions, and overall is detrimental to the community and the movement. Even more so than when people care deeply and disagree on the “correct” licensing terms.

While I am most definitely not a lawyer, and I speak most definitely only for myself and not my employer, let me try to give you a run down of what’s going on here.

First of all, we need to start with a simplification, and handwavey accept that without an explicit license allowing it, the distribution, modification, and integration of source code is not allowed, or at least that’s the way we perceive it in the wider world. And Free Software licenses, more or less permissive, spell out the terms with which distribution and usage are allowed.

It Is But An Example

As far as I can tell, there’s no provision anywhere that source code used in documentation is exempt from these limitations, except insofar as the license on the documentation itself would apply if not otherwise overridden. And that’s how I started engaging with Adafruit: the documentation for most of their CircuitPython libraries provide a lot of useful examples — and as it turns out they were already released with an open-source license (MIT), but that was not obvious when looking at the docs sites themselves. So I convinced them to add SPDX headers to all their source code, including the examples — and now you can read the example and see immediately which license it’s released under. Isn’t that cool?

Unfortunately, sometimes developers are stubborn and find adding two lines to their documentation examples a distraction, and argue against it, making it annoying for others to use their example source code without either infringing the copyright or going the long way to find the right answers.

Websites, PDFs, Books, they are all equal

But this goes to the double for code that is explicitly written only as example material! Let me take a bit of a detour — my wife went through the awesome Python Crash Course a few months ago. While it suffers from a few of the issues I already complained about when it comes to splitting names, the book is fairly well written and has hands-on exercise that provide enough of a stretch to “my first Python project”. In the later parts of the book, one of the long-building exercise is writing a clone of Space Invaders with PyGame, which turned out to be interesting not just for her writing it, but for myself reviewing it as well, as game programming is definitely not a skill I ever spent time acquiring.

Now, remember I said there’s space to stretch? While the book guides you through building the very basic framework for “Alien Invasion” with full code to go with it, it leaves a lot of holes to be filled. Not just the assets (that it pretty much suggests you Google for and find somewhere online, without any discussion on what you can and cannot use — shout out to the Noun Project which I use for my own projects nowadays), but also some of the more advanced gameplay, and a lot of the refactoring — the way you write the game following the book is definitely more aimed at teaching than at maintaining. So when my wife finished with the book, I started showing her examples of how to refactor the code and introduce new features. So while the basic skeleton is the same as the original from the book, the version she ended up with was nearly fully rewritten. And it’s all in a Git repository!

But she has nothing to show for it. The source code in the book does not provide any licensing information. When I reached out to Eric Matthes (the book’s author) on Twitter asking him if he’d consider applying an opensource license to the code, so that she could publish it on her GitHub account to show off to some of her friends – and with an explicit mention that I’d have liked to use it as a base to test out BeeWare projects and see to contribute to some – he said he’d think about it, but that he wouldn’t feel right to release it under a permissive license that would allow someone to take it and sell it on AppStore and similar. So her options are to ignore licensing and publish the code anyway (after all, nobody cares, and I’m sure I can find plenty of people who did exactly that), or to comply with the (lack of) license and keep it for herself, and only show her friends a video of it working. She went for the latter, as we already had a long discussion of copyright when J Salmeron brought up the topic (and dang, we missed the opportunity to shake his hand as we were standing right behind him at the Beast in Black concert in Amsterdam last year!)

Provide It And They Will Build

There is one case that, personally, drained my will to contribute to an ecosystem even more than the example above. After all, Python Crash Course is a great book, and the only really good reason to publish the code is for “bragging rights” — which is not to say it’s not something, but it’s not the end of the world either.

When a commercial vendor is providing you with an extensible ecosystem for you to build upon, but doesn’t play by the same rules, it’s just… disappointing. In this case the issue is with Saleae, the manufacturer of the Logic Pro 16 analyzer I use for a bunch of different things. You may have noticed me providing screenshots off it when talking about fake candles and infrared. As a vendor, Saleae has very good user support: when I complained on Twitter that I wasted two hours chasing ghosts because I didn’t realise I forgot to connect the USB cable to the analyzer, and the software didn’t make it clear enough it was showing me demo garbage, they engaged, asked me what I would have done differently, and delivered the fix in less than a month. That was awesome support.

So where does it go wrong? Well, in June they updated their software to support Python-based extensions for analysis of specific protocols. I was actually interested in adding support for IR decoding to make my life easier in my TV controlling project, and so when they posted that one of their employees built a duty cycle measure tool and posted it on GitHub I was thrilled!

Except… the repository is there, the source code is there, but there is no license. The extension is pretty much a tutorial by itself on how to build what I needed, but it’s coming with no license attached, and as such I can’t use its code as a base for my own extension. And while I could possibly learn from it, it’s also a poison pill… there’s no license, if I copy it too literally, am I infringing copyright? Maybe, who knows? The author says I should «feel free to look, copy and use [his] Logic 2 extensions in any way [I] would like», but that’s not exactly a very comforting statement when you’re contributing while part of a company.

Final Thoughts

Just be yourself (this is pre-recorded). If you do care about Free Software, please take licensing seriously. If you don’t care about Free Software, because you don’t believe in the ideals behind, or you’re just not part of the ecosystem, then I can’t really blame you for disrespecting licenses, but then again if you rely on proprietary software license, you probably should respect all of them. It’s the same problem with software piracy.

I do believe that the folks at REUSE are doing a great service for all of us by making it possible to spell out licenses clearly and openly, and making it easy for others to modify and copy the code that we want to be out there in the world. It doesn’t take so much time to use the tool to add a few lines to a text file, or an additional text file for binary files. Please take the chance to sort this out!

Don’t Ignore Windows 10 as a Development Platform for FLOSS

Important Preface: This blog post was written originally on 2020-05-12, and scheduled for later publication, inspired by this short Twitter thread. As such it well predates Microsoft’s announcement of expanding support of WSL2 to graphical apps. I considered trashing, or seriously re-editing the blog post in the light of the announcement, but I honestly lack the energy to do that now. It left a bad taste in my mouth to know that it will likely get drowned out in the noise of the new WSL2 features announcement.

Given the topic of this post I guess I need to add a preface to point out my “FLOSS creds” — because I have seen already too many attacks to people who even use Windows at all. I have been an opensource developer for over fifteen years now, and part of the reason why I left my last bubble was because it made it difficult for me to contribute to various opensource projects. I say this because I’m clearly a supporter of Free Software and Open Source, wherever possible. I also think that’s different people have different needs, and that ignoring that is a failure of the FLOSS movement as a whole.

The “Year of Linux on the Desktop” is now a meme that has been running its course to the point of being annoying. Despite what FLOSS advocates keep saying, “Linux on the Desktop” is not really moving, and while I do have some strong opinions on this, that’s for another day. Most users, and in particular newcomers to FLOSS (both as users and developers) are probably using a more “user friendly” platform — if you leave a comment with the joke on UNIX being selective with its friends, you’ll end up on a plonkfile, be warned.

About ten years ago, it seemed like the trend was for FLOSS developers to use MacBooks as their daily laptops. I did that for a while myself — an UNIX-based platform with all the tools of the trade, which allowed quite a bit of work being done without having access to a Linux platform. SSH, Emacs, GCC, Ruby, and so on. And at the same time, you had the stability of Mac OS X, with the battery life and all the hardware worked great out of the box. But then more recently, Apple’s move towards “walled gardens” seemed to be taking away from this feasibility.

But back to the main topic. Over the past many years, I’ve been using a “mixed setup” — using a Linux laptop (or more recently desktop) for development, and a Windows (7, then 10) desktop for playing games, editing photos, designing PCBs, and for logic analysis. The latter is because Saleae Logic takes a significant amount of RAM when analysing high-frequency signals, and I have been giving my gamestations as much RAM as I can just for Lightroom, so it makes sense to run it on the machine with 128GB of RAM.

But more recently I have been exploring the ability of using Windows 10 as a development platform. In part because my wife has been learning Python, and since also learning a new operating system and paradigm at the same time would have been a bloody mess, she’s doing so on Windows 10 using Visual Studio Code and Python 3 as distributed through the Microsoft Store. While helping her, I had exposure to Windows as a Python development platform, so I gave it a try when working on my hack to rename PDF files, which turned out to be quite okay for a relatively simple workflow. And the work on the Python extension keeps making it more and more interesting — I’m not afraid to say that Visual Studio Code is better integrated with Python than Emacs, and I’m a long-time user of Emacs!

In the last week I have actually stepped up further how much development I’m doing on Windows 10 itself. I have been using HyperV virtual machines for Ghidra, to make use of the bigger screen (although admittedly I’m just using RDP to connect to the VM so it doesn’t really matter that much where it’s running), and in my last dive into the Libre 2 code I felt the need to have a fast and responsive editor to go through executing part of the disassembled code to figure out what it’s trying to do — so once again, Visual Studio Code to the rescue.

Indeed, Windows 10 now comes with an SSH client, and Visual Studio Code integrates very well with it, which meant I could just edit the files saved in the virtual machine and have the IDE also build them with GCC and executing them to get myself an answer.

Then while I was trying to use packetdiag to prepare some diagrams (for a future post on the Libre 2 again), I found myself wondering how to share files between computers (to use the bigger screen for drawing)… until I realised I could just install the Python module on Windows, and do all the work there. Except for needing sed to remove an incorrect field generated in the SVG. At which point I just opened my Debian shell running in WSL, and edited the files without having to share them with anything. Uh, score?

So I have been wondering, what’s really stopping me from giving up my Linux workstation for most of the time? Well, there’s hardware access — glucometerutils wouldn’t really work on WSL unless Microsoft is planning a significant amount of compatibility interfaces to be integrated. Similar for using hardware SSH tokens — despite PC/SC being a Windows technology to begin with. Screen and tabulated shells are definitely easier to run on Linux right now, but I’ve seen tweets about modern terminals being developed by Microsoft and even released FLOSS!

Ironically, I think it’s editing this blog that is the most miserable experience for me on Windows. And not just because of the different keyboard (as I share the gamestation with my wife, the keyboard is physically a UK keyboard — even though I type US International), but also because I miss my compose key. You may have noticed already that this post is full of em-dashes and en-dashes. Yes, I have been told about WinCompose, but last time I tried using it, it didn’t work and even screwed up my keyboard altogether. I’m now trying it again, at least on one of my computers, and if it doesn’t explode in my face again, I may just give it another try later.

And of course it’s probably still not as easy to set up a build environment for things like unpaper (although at that point, you can definitely run it in WSL!), or to have a development environment for actual Windows applications. But this is all a matter of different set of compromises.

Honestly speaking, it’s very possible that I could survive with a Windows 10 laptop for my on-the-go opensource work, rather than the Linux one I’ve been using. With the added benefit of being able to play Settlers 3 without having to jump through all the hoops from the last time I tried. Which is why I decided that the pandemic lockdown is the perfect time to try this out, as I barely use my Linux laptop anyway, since I have a working Linux workstation all the time. I have indeed reinstalled my Dell XPS 9360 with Windows 10 Pro, and installed both a whole set of development tools (Visual Studio Code, Mu Editor, Git, …) and a bunch of “simple” games (Settlers, Caesar 3, Pharaoh, Age of Empires II HD); Discord ended up in the middle of both, since it’s actually what I use to interact with the Adafruit folks.

This doesn’t mean I’ll give up on Linux as an operating system — but I’m a strong supporter of “software biodiversity”, so the same way I try to keep my software working on FreeBSD, I don’t see why it shouldn’t work on Windows. And in particular, I always found that providing FLOSS software on Windows a great way to introduce new users to the concept of FLOSS — focusing more on providing FLOSS development tools means giving an even bigger chance for people to build more FLOSS tools.

So is everything ready and working fine? Far from it. There’s a lot of rough edges that I found myself, which is why I’m experimenting with developing more on Windows 10, to see what can be improved. For instance, I know that the reuse-tool has some rough edges with encoding of input arguments, since PowerShell appears to still not default to UTF-8. And I failed to use pre-commit for one of my projects — although I have not taken notice yet much of what failed, to start fixing it.

Another rough edge is in documentation. Too much of it assumes only a UNIX environment, and a lot of it, if it has any support for Windows documentation at all, assumes “old school” batch files are in use (for instance for Python virtualenv support), rather than the more modern PowerShell. This is not new — a lot of times modern documentation is only valid on bash, and if you were to use an older operating system such as Solaris you would find yourself lost with the tcsh differences. You can probably see similar concerns back in the days when bash was not standard, and maybe we’ll have to go back to that kind of deal. Or maybe we’ll end up with some “standardization” of documentation that can be translated between different shells. Who knows.

But to wrap this up, I want to give a heads’ up to all my fellow FLOSS developers that Windows 10 shouldn’t be underestimated as a development platform. And that if they intend to be widely open to contributions, they should probably give a thought of how their code works on Windows. I know I’ll have to keep this in mind for my future.

REUSE: Simplifying Code Licensing

I have recently written how licensing is one of the important things to make it easy to contribute code. While I was preparing that blog post, I was also asking Matija if he knew of anything that would validate the presence of license and copyright information in new files. This is because in the past I might have forgotten it myself, and I have definitely merged a pull request or two in which a new contributor forgot to add the headers to the new files — I’m not blaming them, I’m not even blaming myself, I blame the fact that nothing stopped either us!

And indeed, Matija pointed at REUSE, which is a project by FSFE (which I still support, because they are positive!), and in particular at the reuse-tool, which includes a linter, which will ensure that every file in a repository is properly tagged with a license, either inline or (if not possible) through an explicit .license file. I love the idea.

The tool is still a bit rough around the edges, and for instance (because of the spec) it does not have any provision to ignore 0-sized files (or symlinks, as it turns out). Hopefully that can be fixed in the spec and the tool soon. When I started using it, it also didn’t know how to deal with any file that is there to support Autotools, which was clearly something I needed to fix, but that’s a minor issue — clearly the tool has focused on the stuff people care the most about, and Autotools projects are definitely going out of fashion, for good or bad.

I’ve now used reuse-tool to add proper licensing to all the files in most of the repositories that I’ve been actively working on. I say most — I have not touched usbmon-tools yet, because for that one I need to pay more attention, as the copyright not not fully mine. Which means that most likely even the silly configuration files that are unlikely to be copyrightable will have to be licensed under Apache 2.0.

Speaking of the configuration files — the FAQ I linked above suggests using CC0-1.0 license for them. I originally followed that and it took me a moment to remember that I should not do that. The reason is once again found in the previous post: CC0-1.0 is not an OSI-approved license, and that makes it impossible for some people (starting with my ex-colleagues at Google and other Alphabet companies) to contribute to the software, even if it’s just fixing my Travis CI configuration. Instead I selected MIT — which is pretty much equivalent in practice, even though not in theory. Update 2020-05-14: there’s some discussion of alternative recommendations going on right now. Considering that, I have changed my mind and will use Unlicense for configuration files, for the foreseeable future. As I said in the other post, Fedora prefers CC0-1.0, but it does not seem to be outright banned by any organization or project.

I did that for a number of my projects, including those under the python-scsi organization, and included it into a pending pull request I had already open for cachecontrol. Not all of them pass the linter clean yet, because of the 0-sized file issue I noted above. I also didn’t set them up to use pre-commit (despite Carmen adding support for it very quickly), because of that. But at least it’s a step in the right direction, for sure.

Speaking of pre-commit — one of the reasons why I wanted to have this is to make sure that the .license files are not left uncommitted. With the pre-commit check in place, the lint needs to pass for the staged changes rather than for the checked out tree. Once again, yay for automation.

I have to say that this push from FSFE — particularly because I have found myself unable to contribute to one of their projects before, because of missing licensing information on the repository. I also like the fact that they do care about getting people to use this, rather than making a purity tool for the sake of purity, which I’ve seen happening in other organisations. Again, score one for FSFE.

So if you have an open source project, and you want to make sure it’s easy for everyone, including those who may be working for big corporations, to contribute back, give a try to just setting this up with the tool. It should reduce significantly the cost of contribution, and even that of adoption.

Environment and Software Freedom — Elitists Don’t Get It

I have previously complained loudly about “geek supremacists” and the overall elitist stance I have seen in Free Software, Open Source, and general tech circles. This shows up not just in a huge amount of “groupthink” that Free Software is always better, as well as in jokes that may sound funny at first, but are actually trying to exclude people (e.g. the whole “Unix chooses its friends” line).

There’s a similar attitude that I see around environmentalism today, and it makes me uneasy, particularly when it comes to “fight for the planet” as some people would put it. It’s not just me, I’ve seen plenty of acquaintances on Twitter, Facebook, and elsewhere reporting similar concerns. One obvious case is the lack of thought given to inclusion and accessibility: whether it is a thorough attack of pre-peeled oranges with no consideration to those who are not able to hold a knife, or waste-shaming with the infamous waste jars (as an acquaintance reported, and I can confirm the same is true for me, would fill up in a fraction of the expected time just from medicine blisters).

Now the problem is that, while I have expressed my opinions about Free Software and activists a number of times in the past, I have no experience or expert opinion to write a good critique of environmentalist groups, which means I can only express my discomfort and leave it to someone else. Although I wrote about this in the past.

What I can provide some critique of, though, is an aspect that I recently noticed in my daily life, and for which I can report directly, at least for a little bit. And it goes back to the zero-waste topic I mentioned in passing above. I already said that the waste produced just by the daily pills I take (plus the insulin and my FreeStyle Libre sensors) goes beyond what some of the more active environmentalists consider appropriate. Medicine blisters, insulin pens, and the sensors’ applicators are all non-recyclable waste. This means that most of the encouragement to limit waste is unreachable for most people on medications.

The next thing I’m going to say is that waste reduction is expensive, and not inclusive of most people who don’t have a lot of spare disposable cash.

Want a quick example? Take hand wash refills. Most of the people I know use liquid soap, and they buy a new bottle, with a new pump, each time it finishes. Despite ceramic soap bottle being sold in most homeware stores, I don’t remember the last time I saw anyone I know using one. And even when my family used those for a little while, they almost always used a normal soap bottle with the pump. That’s clearly wasteful, so it’s not surprising that, particularly nowadays, there’s a lot of manufacturers providing refills — pouches, usually made with thinner, softer plastic, with a larger amount of soap, that you can use to either refill the original bottles, or to use with one of those “posh” ceramic bottles. Some of the copy on the those pouches explicitly state «These refill pouches use 75% less plastic per ml of product than a [brand] liquid handwash pump (300 ml), to help respect the environment.»

The problem with these refills, at least here in London, is that they are hard to come by, and only a few, expensive brands appear to provide them. For instance you can get refills for L’Occitane hand wash, but despite liking some of their products, at home we are not fond of their hand wash, particularly not at £36 a litre (okay, £32.4 with the recycling discount). Instead we ended up settling on Dove’s hand wash, which you can buy in most stores for £1 for the 250ml bottle (£4/litre). Dove does make refills and sell them, and at least in Germany, Amazon sells them for a lower per-litre price than the bottle. But those refills are not sold in the UK, and if you wanted to order them from overseas they would be more expensive (and definitely not particularly environmentally friendly).

If the refills are really making such a difference as the manufacturers insist they do, they should be made significantly more affordable. Indeed, in my opinion you shouldn’t be able to get the filled bottles alone at all, and they should rather be sold bundled with the refills themselves, at a higher per-liter price.

But price is clearly not the only problem — handwash is something that is subjected to personal taste a lot since our hands are with us all day long. People prefer no fragrance, or different fragrances. The fact that I can find the whopping total of two handwash refills in my usual local stores, that don’t cost more than the filled bottle is not particularly encouraging.

Soap is not the only the thing for which the “environmentally conscious” option is far from affordable. Recently, we stumbled across a store in Chiswick that sells spices, ingredients and household items plastic free, mostly without containers (bring your own, or buy it from them), and we decided to try it, easily since I’ve been saving up the glass containers from Nutella and the jams, and we had two clean ones at home for this.

This needs a bit more context: both me and my wife love spicy food in general, and in particular love mixing up a lot of different spices when making sauces or marinades, which means we have a fairly well stocked spice cupboard. And since we consume a lot of them, we have been restocking them with bags of spices rather than with new bottles (which is why we started cleaning and setting aside the glass jars), so the idea of finding a place where you can fill your own jar was fairly appealing to me. And while we did expect a bit of a price premium given the location (we were in Chiswick after all), it was worth a try.

Another caveat on all of this: the quality, choice and taste of ingredients are not obvious. They are, by definition, up to personal taste. Which means that doing a direct price-by-price comparison is not always possible. But at the same time, we do tend to like the quality of spices we find, so I think we’ve been fair when we boggled at the prices, and in particular at the prices fluctuation between different ingredients. So I ended up making a quick comparison table, based off the prices on their website, and the websites of Morrisons and Waitrose (because, let’s be honest, that’s probably the closest price comparison you want to make, as both options are clearly middle-to-upper class).

Price comparison between Source, Morrisons, Waitrose and the Schwartz brand spices. More accessible on Google Drive.
I’ve taken the cheapest priced option for all the searches, looking for bigger sizes.

If you look at the prices, you can see that, compared with the bottled spices, they are actually fairly competitive! I mean cumin costs over four times if you buy it in bottle at Waitrose, so getting it cheaper is definitely a steal… until you notice that Morrisons stocks a brand (Rajah) that is half the price. Indeed, Rajah appears to sell spices in big bags (100g or 400g), and at a significantly lower price than most of the other options. In personal taste, we love them.

A few exceptions do come to mind: sumac is not easy to find, and it’s actually cheaper at Source. Cayenne pepper is (unsurprisingly) cheaper than Waitrose, and not stocked at Morrisons at all, so we’ll probably pop by again to fill in a large jar of it. Coarse salt is cheaper, and even cheaper than the one I bought on Amazon, but I bought 3Kg two years ago and we still have one unopened bag.

The one part of the pictures that the prices don’t tell, of course, is the quality and the taste. I’ll be very honest and say that I personally dislike the Waitrose extra virgin olive oil I chose the price of (although it’s a decent oil); the Morrisons one is not the cheapest, but that one tasted nasty when I tried it, so I went for the one we actually usually buy. Since we ran out of oil at home, and we needed to buy some anyway, we are now using Source’s and, well, I do like it actually better than Morrisons, so we’ll probably stick to buying it, despite it being more expensive — it’s still within the realm of reasonable prices for good extra virgin olive oil. And they sell it in a refillable bottle, so next time we’ll use that one again.

Another thing that is very clear from the prices is just how much the “organic” label appears to weigh in on the cost of food. I don’t think it’s reasonable to pay four times the price for sunflower oil — and while it is true that I’m comparing the prices of a huge family bottle with that of a fill-your-own-bottle shop, which means you can get less of it at a time, and you pay for that convenience, it’s also one of the more easily stored groceries, so I think it’s fair enough.

And by the way, if you followed my twitter rant, I have good news. Also in Chiswick there’s a Borough Kitchen store, old good brick-and-mortar, and they had a 1L bottle for an acceptable £5.

So where does this whole rant get us? I think that the environment needs for activists to push for affordable efforts. It’s not useful if the zero-waste options are only available to the top 5%. I have a feeling that indeed for some of the better, environmentally aware options we’ll have to pay more. But that should not mean paying £5 for a litre of sunflower oil! We should make sure we can feed the people in the world, if you think that the world is worth saving, and do so in a reasonable way.

Before closing let me just point out the obvious: Source appears to have their heart in the right place with this effort. Having had my own business, I’m sure that the prices reflect the realities of renting a space just off Chiswick High Road, paying for the staff, the required services, the suppliers, and the hidden cost of families with children entering the store and letting their kids nibble on the candies and nuts straight out of the boxes (I’ve seen at least one while we were inside!), without paying or buying anything else.

What I fear we really need is this type of services to scale to the level of big high street grocery stores. Maybe with trade-in containers in place of bring-your-own for deliveries (which I would argue can be more environmentally-friendly than people having to take a car to go grocery shopping). But that’s something I can only hope for.

We need Free Software Co-operatives, but we probably won’t get any

The recent GitHub craze that got a number of Free Software fundamentalists to hurry away from GitHub towards other hosting solutions.

Whether it was GitLab (a fairly natural choice given the nature of the two services), BitBucket, or SourceForge (which is trying to rebuild a reputation as a Free Software friendly hosting company), there are a number of options of new SaaS providers.

At the same time, a number of projects have been boasting (and maybe a bit too smugly, in my opinion) that they self-host their own GitLab or similar software, and suggested other projects to do the same to be “really free”.

A lot of the discourse appears to be missing nuance on the compromises that using SaaS hosting providers, self-hosting for communities and self-hosting for single projects, and so I thought I would gather my thoughts around this in one single post.

First of all, you probably remember my thoughts on self-hosting in general. Any solution that involves self-hosting will require a significant amount of ongoing work. You need to make sure your services keep working, and keep safe and secure. Particularly for FLOSS source code hosting, it’s of primary importance that the integrity and safety of the source code is maintained.

As I already said in the previous post, this style of hosting works well for projects that have a community, in which one or more dedicated people can look after the services. And in particular for bigger communities, such as KDE, GNOME, FreeDesktop, and so on, this is a very effective way to keep stewardship of code and community.

But for one-person projects, such as unpaper or glucometerutils, self-hosting would be quite bad. Even for xine with a single person maintaining just site+bugzilla it got fairly bad. I’m trying to convince the remaining active maintainers to migrate this to VideoLAN, which is now probably the biggest Free Software multimedia project and community.

This is not a new problem. Indeed, before people rushed in to GitHub (or Gitorious), they rushed in to other services that provided similar integrated environments. When I became a FLOSS developer, the biggest of them was SourceForge — which, as I noted earlier, was recently bought by a company trying to rebuild its reputation after a significant loss of trust. These environments don’t only include SCM services, but also issue (bug) trackers, contact email and so on so forth.

Using one of these services is always a compromise: not only they require an account on each service to be able to interact with them, but they also have a level of lock-in, simply because of the nature of URLs. Indeed, as I wrote last year, just going through my old blog posts to identify those referencing dead links had reminded me of just how many project hosting services shut down, sometimes dragging along (Berlios) and sometimes abruptly (RubyForge).

This is a problem that does not only involve services provided by for-profit companies. Sunsite, RubyForge and Berlios didn’t really have companies behind, and that last one is probably one of the closest things to a Free Software co-operative that I’ve seen outside of FSF and friends.

There is of course Savannah, FSF’s own Forge-lookalike system. Unfortunately for one reason or another it has always lagged behind the featureset (particularly around security) of other project management SaaS. My personal guess is that it is due to the political nature of hosting any project over on FSF’s infrastructure, even outside of the GNU project.

So what we need would be a politically-neutral, project-agnostic hosting platform that is a co-operative effort. Unfortunately, I don’t see that happening any time soon. The main problem is that project hosting is expensive, whether you use dedicated servers or cloud providers. And it takes full time people to work as system administrators to keep it running smoothly and security. You need professionals, too — or you may end up like lkml.org being down when its one maintainer goes on vacation and something happens.

While there are projects that receive enough donations that they would be able to sustain these costs (see KDE, GNOME, VideoLAN), I’d be skeptical that there would be an unfocused co-operative that would be able to take care of this. Particularly if it does not restrict creation of new projects and repositories, as that requires particular attention to abuse, and to make good guidelines of which content is welcome and which one isn’t.

If you think that that’s an easy task, consider that even SourceForge, with their review process, that used to take a significant amount of time, managed to let joke projects use their service and run on their credentials.

A few years ago, I would have said that SFLC, SFC and SPI would be the right actors to set up something like this. Nowadays? Given their infights I don’t expect them being any useful.

Can you run a brick and mortar store on Free Software?

I have written before about the CRM I wrote for a pizzeria and I am happy to see that even FSFE started looking into Free Software for SME. I also noted the needs for teams to develop healthy projects. Today I want to give an example of why I think these things are not as easy as most people expect them to be, and how many different moving parts exist that are required to align to make Free Software for SME.

As I’m no longer self-employed, and I have no intention of going back to be a MSP in my lifetime, what I’m writing here is more of a set of “homework pointers” if a community of SME-targeted Free Software projects would be formed.

I decided to focus in my thoughts on the need of a brink and mortar store (or high street store if you prefer), mostly because it has a subset of the requirements that I could think of, compared to a restaurant like the pizza place I actually worked with.

These notes are also probably a lot more scattered and incomplete than I would like, because I have only worked retail for a short while, between high school and the two miserable week of university, nearly fifteen years ago, in a bookstore to be precise.

For most of the people who have not worked retail, it might seem like the most important piece of software/hardware for a store is the till, because that is what they interact with most of the time. While the till systems (also called POS) are fairly important, as those are in direct contact with the customer, they are only the tip of the iceberg.

But let’s start with the POS: whether you plan on integrating them directly with a credit card terminal or not, right now there are a number of integrated hardware/software solution for these, that include a touchscreen to input the receipt components and a (usually thermal) printer for the receipts to be printed on, while sometimes allowing the client to be emailed the receipt instead. As far as I know, there’s no Free Software system for this. I do see an increasing number of Clover tills in Europe, and Square in the United States (but these are not the only ones).

The till software is more complicated than one would think, because in addition to the effects that the customers can see (select line items, print receipt, eventually take payment), it has to be able to keep track of the cash flow, whether it is in form of actual cash, or in the form of card payments. Knowing the cash flow is a requisite for any business, as without that information you cannot plan your budgets.

In bigger operations, this would feed into a dedicated ERP system, which would often include an inventory management software — because you need to know how much stock you have and how fast it is moving, to know when to order new stock.

There is also the need to handle invoices, which usually don’t get printed by the till (you don’t want an invoice printed on thermal paper, particularly in countries like Italy, where you’re meant to keep the original of an invoice for over ten years).

And then there is the filing of payable invoices and, well, their payment. This is part of the accounting procedures, and I know of very few systems that allow integration with a bank to the point of automating this part. PSD2 is meant to require financial institutions to provide APIs to make this possible, at least in Europe, but that has been barely received yet, and we’ll have to see what the solution will be.

Different industries have different expected standards, too. When I worked in the bookstore, there was a standard piece of software that was used to consult the online stock of books from various depots, which was required to handle orders of books for people looking for something that was not in the store. While Amazon and other online services have for the most part removed the need for many to custom order books in a store, I know still a few people who do so, simply to make sure the bookstore stays up. And I assume that very similar, yet different, software and systems exist for most other fields of endeavour, such as computer components, watches, and shoes.

Depending on the size of the store, and the amount of employees, and in general the hours of operation, there may also be need for a roster management software, so that the different workers have fair (and legal) shifts, while still being able to manage days off. I don’t know how well solutions like Workday work for small realities, but in general I feel this is likely going to be one area in which Free Software won’t make an easy dent: following all the possible legal frameworks to actually be compliant with the law is the kind of work that requires a full-time staff of people, and unless something changes drastically, I don’t expect any FLOSS project to keep up with that.

You can say that this post is not giving any answer and is just adding more questions. And that’s the case, actually. I don’t have the time or energy of working on this myself, and my job does not involve working with retailers, or even developing user-focused software. I wanted to write this as a starting point of a project if someone is interested in doing so.

In particular, I think that this would be prime territory for a multi-disciplinary university project, starting from asking questions to store owners of their need, and understanding the whole user journey. Which seems to be something that FSFE is now looking into fostering, which I’m very happy about.

Please, help the answer to the question “Can you run a brink and mortar store on Free Software?” be Yes!

Two words about my personal policy on GitHub

I was not planning on posting on the blog until next week, trying to stick on a weekly schedule, but today’s announcement of Microsoft acquiring GitHub is forcing my hand a bit.

So, Microsoft is acquiring GitHub, and a number of Open Source developers are losing their mind, in all possible ways. A significant proportion of comments on this that I have seen on my social media is sounding doomsday, as if this spells the end of GitHub, because Microsoft is going to ruin it all for them.

Myself, I think that if it spells the end of anything, is the end of the one-stop-shop to work on any project out there, not because of anything Microsoft did or is going to do, but because a number of developers are now leaving the platform in protest (protest of what? One company buying another?)

Most likely, it’ll be the fundamentalists that will drop their projects away to GitHub. And depending on what they decide to do with their projects, it might even not show on anybody’s radar. A lot of people are pushing for GitLab, which is both an open-core self-hosted platform, and a PaaS offering.

That is not bad. Self-hosted GitLab instances already exist for VideoLAN and GNOME. Big, strong communities are in my opinion in the perfect position to dedicate people to support core infrastructure to make open source software development easier. In particular because it’s easier for a community of dozens, if not hundreds of people, to find dedicated people to work on it. For one-person projects, that’s overhead, distracting, and destructive as well, as fragmenting into micro-instances will cause pain to fork projects — and at the same time, allowing any user who just registered to fork the code in any instance is prone to abuse and a recipe for disaster…

But this is all going to be a topic for another time. Let me try to go back to my personal opinions on the matter (to be perfectly clear that these are not the opinions of my employer and yadda yadda).

As of today, what we know is that Microsoft acquired GitHub, and they are putting Nat Friedman of Xamarin fame (the company that stood behind the Mono project after Novell) in charge of it. This choice makes me particularly optimistic about the future, because Nat’s a good guy and I have the utmost respect for him.

This means I have no intention to move any of my public repositories away from GitHub, except if doing so would bring a substantial advantage. For instance, if there was a strong community built around medical devices software, I would consider moving glucometerutils. But this is not the case right now.

And because I still root most of my projects around my own domain, if I did move that, the canonical URL would still be valid. This is a scheme I devised after getting tired of fixing up where unieject ended up with.

Microsoft has not done anything wrong with GitHub yet. I will give them the benefit of the doubt, and not rush out of the door. It would and will be different if they were to change their policies.

Rob’s point is valid, and it would be a disgrace if various governments would push Microsoft to a corner requiring it to purge content that the smaller, independent GitHub would have left alone. But unless that happens, we’re debating hypothetical at the same level of “If I was elected supreme leader of Italy”.

So, as of today, 2018-06-04, I have no intention of moving any of my repositories to other services. I’ll also use a link to this blog with no accompanying comment to anyone who will suggest I should do so without any benefit for my projects.

The importance of teams, and teamwork

Today, on Twitter, I have received a reply with a phrase that, in its own sake and without connecting back with the original topic of the thread, I found significant of the dread I feel with working as a developer, particularly in many opensource communities nowadays.

Most things don’t work the way I think they work. That’s why I’m a programmer, so I can make them work the way I think they should work.

I’m not going to link back to the tweet, or name the author of the phrase. This is not about them in particular, and more about the feeling expressed in this phrase, which I would have agreed with many years ago, but now feels so much off key.

What I feel now is that programmers don’t make things work the way they think they should. And this is not intended as a nod to the various jokes about how bad programming actually is, given APIs and constraints. This is about something that becomes clear when you spend your time trying to change the world, or make a living alone (by running your own company): everybody needs help, in the form of a team.

A lone programmer may be able to write a whole operating system (cough Emacs), but that does not make it a success in and by itself. If you plan on changing the world, and possibly changing it for the better, you need a team that includes not only programmers, but experts in quite a lot of different things.

Whether it is a Free Software project, or a commercial product, if you want to have users, you need to know what they want — and a programmer is not always the most suitable person to go through user stories. Hands up all of us who have, at one point or another, facepalmed at an acquaintance taking a screenshot of a web page to paste it into Word, and tried to teach them how to print the page to PDF. While changing workflows so that they make sense may sound the easiest solution to most tech people, that’s not what people who are trying to just do their job care about. Particularly not if you’re trying to sell them (literally or figuratively) a new product.

And similarly to what users want to do, you need to know what the users need to do. While effectively all of Free Software comes with no warranty attached, even for it (and most definitely for commercial products), it’s important to consider the legal framework the software has to be used on. Except for the more anarchists of the developers out there, I don’t think anyone would feel particularly interested in breaching laws for the sake of breaching them, for instance by providing a ledger product that allows “black book accounting” as an encrypted parallel file. Or, to reprise my recent example, to provide a software solution that does not comply with GDPR.

This is not just about pure software products. You may remember, from last year, the teardown of Juicero. In this case the problems appeared to step by the lack of control over the BOM. While electronics is by far not my speciality, I have heard more expert friends and colleagues cringe at seeing the spec of projects that tried to actually become mainstream, with a BOM easily twice as expensive as the minimum.

Aside here, before someone starts shouting about that. Minimising the BOM for an electronic project may not always be the main target. If it’s a DIY project, making it easier to assemble could be an objective, so choosing more bulky, more expensive parts might be warranted. Similarly if it’s being done for prototyping, using more expensive but widely available components is generally a win too. I have worked on devices that used multi-GB SSDs for a firmware less than 64MB — but asking for on-board flash for the firmware would have costed more than the extremely overprovisioned SSDs.

And in my opinion, if you want to have your own company, and are in for the long run (i.e. not with startup mentality of getting VC capital and get acquired before even shipping), you definitely need someone to follow up the business plan and the accounting.

So no, I don’t think that any one programmer, or a group of sole programmers, can change the world. There’s a lot more than writing code, to build software. And a lot more than building software, to change society.

Consider this the reason why I will plonk-file any recruitment email that is looking for “rockstars” or “ninjas”. Not that I’m looking for a new gig as I type this, but I would at least give thought if someone was looking for a software mechanic (h/t @sysadmin1138).

Diabetes Software: the importance of documented protocols

You may remember that just last week I was excited to announce that I had more work planned and lined up for my glucometer utilities, one of which was supporting OneTouch Verio IQ which is a slightly older meter that is still sold and in use in many countries, but for which no protocol is released.

In the issue I linked above you can find an interesting problem: LifeScan discontinued their Diabetes Management Software, and removed it from their website. Indeed instead they suggest you get one of their Bluetooth meters and to use that with their software. While in general the idea of upgrading a meter is sane, the fact that they decided to discontinue the old software without providing protocols is at the very least annoying.

This shows the importance of having open source tools that can be kept alive as long as needed, because there will be people out there that still rely on their OneTouch Verio IQ, or even on the OneTouch Ultra Easy, which was served by the same software, and is still being sold in the US. Luckily at least they at least used to publish the Ultra Easy protocol specs and they are still available on the Internet at large if you search for them (and I do have a copy, and I can rephrase that into a protocol specification if I find that’s needed).

On the bright side, the Tidepool project (of which I wrote before) has a driver for the Verio IQ. It’s not a particularly good driver, as I found out (I’ll get to that later), but it’s a starting point. It made me notice that the protocol was almost an in-between of the Ultra Easy and the Verio 2015, which I already reverse engineered before.

Of course I also managed to find a copy of the LifeScan software on a mostly shady website and a copy of the “cable drivers” package from the Middle East and Africa website of LifeScan, which still has the website design from five years ago. This is good because the latter package is the one that installs kernel drivers on Windows, while the former only contains userland software, which I can trust a little more.

Comparing the USB trace I got from the software with the commands implemented in the TidePool driver showed me a few interesting bits of information. The first being the first byte of commands on the Verio devices is not actually fixed, but can be chosen between a few, as the Windows software and the TidePool driver used different bytes (and with this I managed to simplify one corner case in the Verio 2015!). The second is that the TidePool driver does not extract all the information it should! In particular the device allows before/after meal marking, but they discard the byte before getting to it. Of course they don’t seem to expose that data even from the Ultra 2 driver so it may be intentional.

A bit more concerning is that they don’t verify that the command returned a success status, but rather discard the first two bytes every time. Thankfully it’s very easy for me to check that.

On the other hand, reading through the TidePool driver (which I have to assume was developed with access to the LifeScan specifications, under NDA) I could identify two flaws in my own code. The first was not realizing the packet format between the UltraEasy and the Verio 2015 was not subtly different as I thought, but it was almost identical, except the link-control byte in both Verio models is not used, and is kept to 0. The second was that I’m not currently correctly dropping out control solutions from the readings of the Verio 2015! I should find a way to get a hold of the control solution for my models in the pharmacy and make sure I get this tested out.

Oh yeah, and the TidePool driver does not do anything to get or set date and time; thankfully the commands were literally the same as in the Verio 2015, so that part was an actual copy-paste of code. I should probably tidy up a bit, but now I would have a two-tier protocol system: the base packet structure is shared between the UltraEasy, Verio IQ and Verio 2015. Some of the commands are shared between UltraEasy and Verio IQ, more of them are shared between the Verio IQ and the Verio 2015.

You can see now why I’ve been disheartened to hear that the development of drivers, even for open source software, is done through closed protocol specifications that cannot be published (or the drivers thoroughly commented). Since TidePool is not actually using all of the information, there is no way to tell what certain bytes of the responses represent. And unless I get access to all the possible variants of the information, I can’t tell how some bytes that to me look like constant should represent. Indeed since the Verio 2015 does not have meal information, I assumed that the values were 32-bit until I got a report of invalid data on another model which shares the same protocol and driver. This is why I am tempted to build “virtual” fakes of these devices with Facedencer to feed variants of the data to the original software and see how it’s represented there.

On the bright side I feel proud of myself (maybe a little too much) for having spent the time to rewrite those two drivers with Construct while at 34C3 and afterwards. If I hadn’t refactored the code before looking at the Verio IQ, I wouldn’t have noticed the similarities so clearly and likely wouldn’t have come to the conclusion it’s a shared similar protocol. And no way I could have copy-pasted between the drivers so easily as I did.

Public Money, Public Code

Imagine that all publicly funded software were under a free license: Everybody would be able to use, study, share and improve it.

I have been waiting for Free Software Foundation Europe to launch the Public Money, Public Code campaign for almost a year now, when first Matthias told me about this being in the works. I have been arguing the same point, although not quite as organized, since back in 2009 when I complained about how the administration of Venice commissioned a GIS application to a company they directly own.

For those who have not seen the campaign yet, the idea is simple: software built with public money (that is, commissioned and paid for by public agencies), should be licensed using a FLOSS license, to make it public code. I like this idea and will support it fully. I even rejoined the Fellowship!

The timing of this campaign ended up resonating with a post on infrastructure projects and their costs, which I find particularly interesting and useful to point out. Unlike the article that is deep-linked there, which lamented of the costs associated with this project, this article focuses on pointing out how that money actually needs to be spent, because for the most part off the shelf Free Software is not really up to the task of complex infrastructure projects.

You may think the post I linked is overly critical of Free Software, and that it’s just a little rough around the edges and everything is okay once you spend some time on it. But that’s exactly what the article is saying! Free Software is a great baseline to build complex infrastructure on top of. This is what all the Cloud companies do, this is what even Microsoft has been doing in the past few years, and it is reasonable to expect most for-profit projects would do that, for a simple reason: you don’t want to spend money working on reinventing the wheel when you can charge for designing an innovative engine — which is a quite simplistic view of course, as sometimes you can invent a more efficient wheel indeed, but that’s a different topic.

Why am I bringing this topic up together with the FSFE campaign? Because I think this is exacly what we should be asking from our governments and public agencies, and the article I linked shows exactly why!

You can’t take off the shelf FLOSS packages and have them run a whole infrastructure, because they usually they are unpolished, and might not scale or require significant work to bring them up to the project required. You will have to spend money to do that, and maybe in some cases it will be cheaper to just not use already existing FLOSS projects at all, and build your own new, innovative wheel. So publicly funded projects need money to produce results, we should not complain about the cost1, but rather demand that the money spent actually produces something that will serve the public in all possible ways, not only with the objective of the project, but also with any byproduct of it, which include the source code.

Most of the products funded with public money are not particularly useful for individuals, or for most for-profit enterprises, but byproducts and improvements may very well be. For example, in the (Italian) post I wrote in 2009 I was complaining about a GIS application that was designed to report potholes and other roadwork problems. In abstract, this is a way to collect and query points of interests (POI), which is the base of many other current services, from review sites, to applications such as Field Trip.

But do we actually care? Sure, by making the code available of public projects, you may now actually be indirectly funding private companies that can reuse that code, and thus be jumpstarted into having applications that would otherwise cost time or money to build from scratch. On the other hand, this is what Free Software has been already about before: indeed, Linux, the GNU libraries and tools, Python, Ruby, and all those tools out there are nothing less than a full kit to quickly start projects that a long time ago would have taken a lot of money or a lot of time to start.

You could actually consider the software byproducts of these project similarly to the public infrastructure that we probably all take from granted: roads, power distribution, communication, and so on. Businesses couldn’t exist without all of this infrastructure, and while it is possible for a private enterprise to set out and build all the infrastructure themselves (road, power lines, fiber), we don’t expect them to do so. Instead we accept that we want more enterprises, because they bring more jobs, more value, and the public investment is part of it.

I actually fear the reason a number of people may disagree with this campaign is rooted in localism — as I said before, I’m a globalist. Having met many people with such ideas, I can hear them in my mind complaining that, to take again the example of the IRIS system in Venice, the Venetian shouldn’t have to pay for something and then give it away for free to Palermo. It’s a strawman, but just because I replaced the city that they complained about when I talked about my idea those eight years ago.

This argument may make sense if you really care about local money being spent locally and not counting on any higher-order funding. But myself I think that public money is public, and I don’t really care if the money from Venice is spent to help reporting potholes in Civitella del Tronto. Actually, I think that cities where the median disposable income is higher have a duty to help providing infrastructure for the smaller, poorer cities at the very least in their immediate vicinity, but overall too.

Unfortunately “public money” may not always be so, even if it appears like that. So I’m not sure if, even if a regulation was passed for publicly funded software development to be released as FLOSS, we’d get a lot in form of public transport infrastructure being open sourced. I would love for it to be though: we’d more easily get federated infrastructure, if they would share the same backend, and if you knew how the system worked you could actually build tools around it, for instance integrating Open Street Map directly with the transport system itself. But I fear this is all wishful thinking and it won’t happen in my lifetime.

There is also another interesting point to make here, which I think I may expand upon, for other contexts, later on. As I said above, I’m all for requiring the software developed with public money to be released to the public with a FLOSS-compatible license. Particularly one that allows using other FLOSS components, and the re-use of even part of the released code into bigger projects. This does not mean that everybody should have a say in what’s going on with that code.

While it makes perfect sense to be able to fix bugs and incompatibilities with websites you need to use as part of your citizen life (in the case of the Venetian GIS I would probably have liked to fix the way they identified the IP address they received the request for), adding new features may actually not be in line with the roadmap of the project itself. Particularly if the public money is already tight rather than lavish, I would surely prefer that they focused on delivering what the project needs and just drop the sources out in compatible licenses, without trying to create a community around them. While the latter would be nice to have, it should not steal the focus on the important part: a lot of this code is currently one-off and is not engineered to be re-used or extensible.

Of course on the long run, if you do have public software available already as open-source, there would be more and more situations where solving the same problem again may become easier, particularly if an option is added there, or a constant string can become a configured value, or translations were possible at all. And in that case, why not have them as features of a single repository, rather than have a lot of separate forks?

But all of this should really be secondary, in my opinion. Let’s focus on getting those sources, they are important, they matter and they can make a difference. Building communities around this will take time. And to be honest, even making these secure will take time. I’m fairly sure that in many cases right now if you do take a look at the software that is running for public services, you can find backdoors, voluntary or not, and even very simple security issues. While the “many eyes” idea is easily disproved, it’s also true that for the most part those projects cut corners, and are very difficult to make sure to begin with.

I want to believe we can do at least this bit.


  1. Okay, so there are case of artificially inflated costs due to friends-of-friends. Those are complicated issues, and I’ll leave them to experts. We should still not be complaining that these projects don’t appear for free.
    [return]