FastMail 9 Months Review

You may remember that earlier this year, for reasons weakly linked to my change of employer, I have been looking to change my email provider. After many years using Gmail via what is now called Google Workspace, the complete disconnection to consumer features including Google One got to me, and I looked for an alternative.

At the time I considered both ProtonMail and FastMail as valid options — the former having been touted all over the place for their security features and privacy. After trying them both, and originally choosing the former, I found out that it would just not scale for the amount of email I had in Gmail, and instead switched over to FastMail, that seemed to fit much better for my needs.

Now it’s over nine months later, and I thought it’s a good time to check-in with my choice, and provide an update for the next person looking to choose an email provider. But before I get into the details, I think it’s also important to qualify that this year has been anything but “business as usual” — not just because of the lockdown, but also because me and my wife ended up moving to a new apartment, and oh, yeah I changed jobs. This means that my email usage pattern changed considerably, and that changed which features and values of FastMail I stressed on.

So let’s start with the elephant in the room, that is the offline availability — one of the things I noted earlier on is that the default Android app of FastMail is not much more than a WebView on a portable web app. It might not be technically precise, but I still feel it’s a good description for it. It does have some offline features, but you can’t “sync offline” a whole folder of email to have access to without a network connection. This would have probably been a much bigger, if not even deal-breaker, problem… if we had been traveling, or if I had been commuting two hours a day every day. But given the lockdown, this part is not really affecting me, nearly at all.

I guess that if this is a significant enough feature you need, using a “proper” client on the phone would be a good option — and that might have been something I’d have tried in different circumstances. As far as I can tell, FastMail does not implement OAuth2-based IMAP login flows, so you still need application-specific passwords for this to work, which I’m not really fond of — if they did, and they also supported U2F from phones (why is it not supported is not clear to me), that would be a total non-issue. As it is, it’s very low in my priorities though, so I’m not complaining.

Let’s segue that into security — as I said I’m still not sure why U2F is not supported on phones, and why I had to enable SMS 2FA to login on my Android phone. But on the other hand, it works fine on computers, so there’s that going on for us. FastMail is also the first provided that I see taking Application-Specific Passwords to the letter: when you create the password you can decide which scopes to attach to it, so if you want a send-only password, you can. And that makes life easier, and not just for developers needing to send kernel patches out.

So there’s space for improvement (OAuth2 and U2F, folks?), but otherwise I’m feeling confident that FastMail knows what they are doing and are taking compromises intentionally, rather than by coincidence.

On the Web UI side, things may be a bit bumpy — the UI feels “less fresh” than Gmail or Inbox, but that’s not entirely bad: it’s definitely lighter to use, and it makes searches work nicely. I think what I am missing from it is a more “native feeling” — I miss the contextual menu to archive or delete threads. The drag-and-drop works great though, so there’s that about it. Some of the choices of where to find actions (such as report spam/report non-spam) are a bit confusing, but otherwise it’s all very serviceable, and very customizable, too! The (optional) Gravatar integration is also fairly handy when talking with people in the FLOSS community, although I wish more services decided to provide a “service identity”.

If anything, there are two features I really miss from Gmail: the first is something that FastMail appears to have tried, and then given up on (for now), and that is the portable web app mode on desktop. For a while I had a FastMail window rather than a tab, and that was easier to handle for me. Having notifications running in the background without needing to have the tab open in my browser just make my flow easier (which is why I do that with Twitter, Home Assistant, and Google Maps among the others), and FastMail would be an obvious option there.

The second feature is the composer “pop up”. When having to deal with the horrible old property management company for the flat we lived in, I often had to go and refer back to previous threads, because they rarely would follow-up in writing, and the reference person changed all the time. I ended up having to open a second (or sometimes third) window open with the various searches, while composing a long message in the first one.

But otherwise? I haven’t been missing Gmail at all, despite the significantly higher amount of email I had to read and write due to the work and flat changes. Filters work fine, the self-deleting folders (or labels, you can choose which version you want!) are an awesome tool to keep the sheer amount of email under control.

Okay, sure, the “Smart Mail” features that I never had would have been nice at times — fill in the calendar with events just by receiving them! But as it turns out, GSuite/Google Workspace never got that feature in for their users. It’s long been a consumers-only feature, and so I never got into the habit of relying on it. And even when I had access to it, the fact that it required explicit enrolling of the providers by Google, it meant that only those services that had enough critical mass, or were used by a Googler, would be covered. It would be so much better if instead there would be better support for providers to attach, say, ICS files when they send you time-based information, such as expected delivery (for groceries, online shopping), or appointments (for hospitals and so on — although this is unlikely to happen: while NHS is very happy to remind me of all appointments in full by SMS, email messages only contains a “Please click here and put some obvious information about you to see the medical letter with the appointment”, alas.)

So, I clearly am a happy customer; even more so now that I have experience Outlook 365 (I can’t make heads or tails of that interface!) And I would say that, if you’re looking for an email-only provider, FastMail is the best option I have seen. Depending on what you’re looking for, Google Workspace might still a better value for money (with the whole collaboration suite), but I have a couple of former customers that I would probably have signed up on FastMail rather than Google, if they asked right now.

To close up, yes, there’s a referral programme nowadays — if you’re interested, this is my affiliate link. You’re welcome to use it, but it’s not the reason why I’m suggesting to use FastMail. I’m suggesting it because it is a terrific service.

You can’t program the world to suit you

Last year, I was drafting more notes regarding the Free Software for SMB that I talked about before. While doing so I recognized that one of the biggest takeaway for myself is that successfully making a software project thrive takes a lot more than just good programmers, developers, designers. If you have customers you need people who know how to make a business work, you need people who can market your product, and you need people to remind you what the customers actually want as well as what they need.

It’s not an entirely new lesson — I wrote (in Italian) about helping Free Software without being a programmer fifteen years ago. I also wrote about the importance of teamwork two years ago. And I have spent a good chunk of my opensource and professional careers knee-deep in documentation matters.

I actually want to go back to the tweet that spawned the teamwork post:

Most things don’t work the way I think they work. That’s why I’m a programmer, so I can make them work the way I think they should work.

This is not meant to single out the author of the quoted phrase, but just to take it as an example of a feeling I get from many talks, and discussions, and in general just people out there. The idea that you can tech your way out of a problem. That by being a programmer you can change the way most things work.

And that’s not true, because the world is not running on servers, unless you found the Repository and I don’t know that. Indeed wielding “the power of programming”, thinking of changing the world just with that, sounds to me like a recipe for either failure or disaster.

I heard all kind of possible “solutions” to this — from insisting on teaching ethics in Software Engineering courses (with reasonable doubts about it), to regulating the heck out of any action businesses can take. I think the closest I have seen to something I would like (with all my biases of course) would be to make sure there is a mix of non-programming subjects in every university or high school that teaches programming as well. But even that has its own limitations, and I can totally say that I would probably have been frustrated by that and just ignored everything that’s not programming-related, when I was that age.

To make the example of Italy, that is under political turmoils most of the time, I could see a number of critiques of (in my opinion horrible) politicians based on where they went to school. In particular I saw some left-wing intellectuals criticising ministers (who have enough to be criticised about in deeds) based on the fact that they didn’t study in a lyceum but rather on a technical (or professional) school. Well, turns out I studied at a tech school, and I studied basic economics and (very basic) civic education for two years, and I found out the hard way that I know how VAT works much better than most of my local acquaintances who got an university degree after a lyceum: they never were introduced to the concept of VAT, the difference between types of taxes, and so on.

You could argue that there is no reason to know this particular tidbit, which is where I’m actually going to end up: there is no perfect education, the same way as there is no perfect solution. People need to learn to work with each other and they should know how to play each other’s strengths instead.

What I really would like to see proposed more often is focusing a lot more on teamwork. And not in the sense of “Here’s a topic for research, now work on it with your team”, which I had to do in high school — many of us have had the experience of being the only person working for a group assignment. What I would have loved to have would be cross-school year-long projects. Not competitions, but rather something that requires more than one type of expertise: trying to get three programming students in a room to work together, in my experience, turned to either two of them slacking off, because one of them actually enjoy doing the work, or if you’re lucky having someone with actual leadership skills telling them how to do their job… but still gives the impression that you just need programmers to do something like that.

In hindsight I would have loved instead if I had a project shared with some of my colleagues from electronics, mechanical and business tech-schools. Come up with a solution for a problem, that requires hardware and software, and a product plan that would include optimising the bill of material for small batch production and still make profits.

Sounds complicated? It is. Having had my own company, alone, for four years, made it very clear that there is a lot more than just being a programmer if you want to succeed. If you want to change the world, and in particular if you want to make the world a better place, then it takes even more energy, and a bigger group of people who can work together.

It also takes leadership. And that’s not something that I feel can be taught, and it’s the one that makes the most difference on whether the change is for good or not. I’m not good at leading people. I don’t have the right mindset most likely. I have trouble rallying people towards a common goal. I know that. I just hope that at some point, when I’ll be looking at more meaning in my work, I’ll find the right leader that can take what I can add to a good team, and let me shine through that.

I know it’s going to be repeating myself, but that is also what I mean with “there is no perfect solution”. If we decided that leadership is something that is important to score people, whether it is with school results, or with performance review at work, then we would be pretty much excluding a significant part of the population: not everyone wants to be a leader, are people who don’t want to be a leader worth less to society? Hint: this is not far from the question of how many multiples of a line worker a CEO should be worth.

And if you need a proper example of how “tech will not solve it”, just look at 2020 in general: tech is not really solving the Covid-19 world crisis. It does help, of course: videopresence, social network and chat services (including my employer’s), online “tabletop” games, shared documents infrastructure, online shopping, and so on… they all allowed people, isolating or not, to feel closer together. But it did not solve the problem. Even if we including medical sciences as “tech”, they still have not managed to find a way to deal with the crisis, because the crisis is not just medical.

People don’t ignore the lockdown requirements because they don’t have enough tech: it’s because there are other things in this world! It’s one thing to talk to my mother on the big screen of Portal, and another thing to spend a week at her house — including the fact that I can’t fix her house’s wiring while physically in another country. And then there is the big elephant in the room: the economy — tech can’t solve that problem, people working in industries that had to shut down because of the lockdown can’t just be “teched” into new roles; they can’t magically be vaccinated overnight; they need political leaders to make tough decisions around supporting them.

So no, you can’t program the world to suit your needs. Great for you if you have more tools in your toolbox – and there’s a lot more use for even basic programming literacy that has nothing to do with working as a programmer – but that doesn’t make you super-human, nor it allows you to ignore what’s going on in the world. If “being a programmer” is providing a superiority complex, I feel it’s more related to the fact that we’ve been well paid for a number of years now, and money makes the difference.

But that’s a topic for an entirely new rant, later on.

Home Assistant and CGG1 Sensors

You may or may not know that I’m one of the few tech people who don’t seem to scream against IoT and I actually have quite a few “smart home” devices in my apartment — funnily enough, one less now that we don’t have normal heating at home and we had to give up the Nest Thermostat. One of the things that I have not set up until now was Home Assistant, simply because I didn’t really need anything from it. But when we moved to this flat, I felt the need to have some monitoring of humidity and temperature in the “winter garden” (a part of the flat that is next to the outside windows, and is not heated), since we moved up in floors and we no longer have the “repair” of being next to another building.

After talking with my friend Srdjan, who had experience with this, I settled on ordering Xiaomi-compatible “ClearGrass” CGG1 sensors from AliExpress. These are BLE sensors, which mean they can broadcast their readings without requiring active connections, and they have a nice eInk display, which wastes very little power to keep running. There’s other similar sensors, some cheaper in the short run, but these sounded like the right compromise: more expensive to buy, but cheaper to run (the batteries are supposed to last six months, and cost less than 50p).

Unfortunately, getting them to work turned out to be a bit more complicated than either of us planned at the beginning. The sensors arrived on a day off, less than two weeks after ordering (AliExpress is actually fairly reliable when ordering to London), and I spent pretty much the whole day, together with Srdjan over on videocall, to get them to work. To save this kind of pain for the next person who come across these issues, I decided to write this up.

Hardware Setup Advices

Before we dig into the actual configuration and setup, let me point out a few things about setting up these devices. The most annoying part is that the batteries are, obviously, isolated to avoid running out in shipping, but the “pull out tab” doesn’t actually pull out. It took quite a bit of force to turn the battery compartment door, open it up, and then re-set it in. Be bold in taking them out.

The other advice is not from me but from Srdjan: write the MAC address (or however it’s called in Bluetooth land) on each of the sensors. Because if you only have one sensor, it’s very easy to tell which one it is, but if you bought more (say, four like I did), then you may have issues later to identify which one is which. So it’s easier to do while you’re setting them up, by turning them on one at a time.

To tell what’s the address, you can either use an app like Beacon Simulator, and listen to the broadcast, or you can wait until you get to a point when you’re ready to listen to the broadcast, later in the process. I would recommend the former, particularly if you live in a block of flats. Not only there’s a lot of stuff that broadcast BLE beacons, but nowadays pretty much every phone is broadcasting them as well due to the Covid 19 Exposure Notifications.

And to make sure that you don’t mix them up, I definitely suggest to use a label maker — and if you are interested in the topic, make sure to check out the Museum of Curiosity Series 13, Episode 6.

Finally, there’s a bit of a spoiler of where this whole process is going to end up going to — I’m going to explicitly suggest you avoid using USB Bluetooth dongles, and instead get yourself an ESP32 kit of some kind. ESP32 devkits are less than £10 on Amazon at the time of writing, and you can find them even cheaper on AliExpress — and they will be much more useful, as I’ll go ahead and tell you.

Home Assistant Direct Integration

So first of all, we spent a lot of time mucking around with the actual Home Assistant integrations. There’s a mitemp_bt sensor integration in Home Assistant, but it’s an “active” Bluetooth implementation, that is (supposedly) more power hungry, and require associating the sensors with the host running Home Assistant. There’s also an alternative implementation, that is supposed to use passive BLE scans.

Unfortunately, even trying to install the alternative implementation turned out to be annoying and difficult — the official instructions appears to expect you install another “store-like” interface on top of Home Assistant, which appears to not be that easy to do when you use their “virtual appliance” image in the first place. I ended up hacking it up a bit, but got absolutely nothing out of it: there isn’t enough logging to know what’s going on at any time, and I couldn’t tell if any packet was even received and parsed.

There is also a clear divergence between the Home Assistant guidelines on how to build new integration, and the way the alternative implementation is written — one of the guides (which I can’t now find easily, and that might speak to the reason for this divergence) explicitly suggests not to write complex parsing logic in the integration, and instead build an external Python library to implement protocols and parsers. This is particularly useful when you want to test something outside of Home Assistant, to confirm it works first.

In this case, having a library (and maybe a command line tool for testing) would have made it easier to figure out if the problem with the sensors was that nothing was received, or that something was wrong with the received data.

This was made more annoying too by the fact that for this to work, you need a working Bluetooth adapter connected to your Home Assistant host — which in my case is a virtual machine. And the alternative implementation tells you that it might interfere with other Bluetooth integrations, so you’re suggested to keep multiple Bluetooth interfaces, one for each of the integrations.

Now this shouldn’t be too hard, but it is: the cheapest Bluetooth dongles I found on Amazon are based on Realtek chipsets, which while supported by (recent enough) Linux kernels, need firmware files. Indeed the one dongle I got requires Linux 5.8 or later, or it requests the wrong firmware file altogether. And there’s no way to install firmware files in the Home Assistant virtual appliance. I tried, quite a few times by now.

ESPHome Saves The Day

ESPHome is a project implementing firmware (or firmware building blocks, rather) for ESP8266 and ESP32 boards and devices, that integrates fairly easily with Home Assistant. And since ESP32 supports BLE, ESPHome supports Xiaomi-compatible BLE sensors, such as the CGG1. So the suggestion from Srdjan, which is what he’s been doing himself, is to basically use an ESP32 board as a BLE-to-WiFi bridge.

This was easy because I had a bunch of ESP32 boards in a box from my previous experiments with acrylic lamps, but as I said they are also the same price, if not cheaper, than Bluetooth dongles. The one I’m using is smaller than most breadboard-compatible ESP32 boards and nearly square — it was a cheap option at the time, but I can’t seem to find one available to build now. It’s working out well by size, because it also doesn’t have any pin headers soldered, so I’m just going to double-side-tape it to something and give it a USB power cable.

But it couldn’t be as easy as to follow the documentation, unfortunately. While configuring the ESPhome is easy, and I did manage to get some readings almost right away, I found that after a couple of minutes, it would stop seeing any signal whatsoever from any of the sensors.

Digging around, I found that this was not uncommon. There’s two ESPhome issues from 2019: #317 and #735 that report this kind of problems, with no good final answer on how to solve them, and unfortunately locked to collaborators, so I can’t leave breadcrumbs for the next person in there — and it’s why I am now writing this, hopefully it’ll save headaches for others.

The problem, as detailed in #735, is that the BLE scan parameters need to be adjusted to avoid missing the sensors’ broadcasts. I tried a few combinations, and at the end found that disabling the “active” scan worked — that is, letting the ESP32 passively listen to the broadcasts, without trying to actively scan the Bluetooth channels seemed to let it stay stable, now for over 24 hours. And it should also be, as far as I can tell, less battery-draining.

The final configuration looks something like this:

esp32_ble_tracker:
  scan_parameters:
    duration: 300s
    window: 48ms
    interval: 64ms
    active: False

sensor:
  - platform: xiaomi_cgg1
    mac_address: "XX:XX:XX:XX:XX:XX"
    temperature:
      name: "Far Corner Temperature"
    humidity:
      name: "Far Corner Humidity"
    battery_level:
      name: "Far Corner Battery Level"

The actual scan parameters should be, as far as I can tell, ignored when disabling the active scan. But since it works, I don’t dare to touch it yet. The code in ESPhome doesn’t make it very clear if changing those parameters when disabling active scan is entirely ignored, and I have not spent enough time going through the deep stack to figure this out for certain.

The only unfortunate option of having it set up this way, is that by default, Home Assistant will report all of the sensors in the same “room”, despite them being spread all over the apartment (okay, not all over the apartment in this case but in the winter garden).

I solved that by just disabling the default Lovelace dashboard and customizing it. Because turns out it’s much nicer to customize those dashboards than using the default, and it actually makes a lot more sense to look at it that way.

Looking To The Future

So now I have, for the first time, a good reason to use Home Assistant and to play around with it a bit. I actually have interesting “connected home” ideas to go with it — but they mostly rely on getting the pantograph windows working, as they don’t seem currently to be opening at all.

If I’m correct in my understanding, we’ll need to get the building managers to come and fix the electronic controls, in which case I’ll ask for something I can control somehow. And then it should be possible to have Home Assistant open the windows in the morning, assuming it’s not raining and the temperature is such that a bit of fresh air would be welcome (which is most of the summer, here).

It also gives me a bit more of an incentive to finish my acrylic lamps work — it would be easy, I think, to use one of those as a BLE bridge that just happens to have a NeoPixel output channel. And if I’m going “all in” to wire stuff into Home Assistant, it would also allow me to use the acrylic lamps as a way to signal for stuff.

So possibly expect a bit more noise on this front from me, either here or on Twitter.

RPC Frameworks and Programming Languages

RPC frameworks are something that I never thought I would be particularly interested in… until I joined the bubble, where nearly everything used the same framework, which made RPC frameworks very appealing. But despite both my previous and current employers releasing two similar RPC frameworks (gRPC and Apache Thrift respectively), they are not really that commonly used in Open Source, from what I can tell. D-Bus technically counts, but it’s also a bus messaging system, rather than a simpler point-to-point RPC system.

On the proprietary software side, RPC/IPC frameworks have existed for dozens of years: CORBA was originally specified in 1991, and Microsoft’s COM was released in 1993. Although these are technically object models rather than just RPC frameworks, they fit into the “general aesthetics” of the discussion.

So, what’s the deal with RPC frameworks? Well, in the general sense, I like to represent them as a set of choices already made for you: they select an IDL (Interface Description Language), they provide some code generation tool leveraging the libraries they select, and they decide how structures are encoded on the wire. They are, by their own nature, restrictive rather than flexible. And that’s the good thing.

Because if we considered the most flexible options, we’d be considering IP as an RPC framework, and that’s not good — if all we have is IP, it’s hard for two components developed in isolation to be able to talk together. That’s why we have higher level protocols, and that’s why even just using HTTP as an RPC protocol is not good enough: it doesn’t define anywhere close to the semantics you need to be able to use it as a protocol without knowing both client and server code.

And one of the restrictions that I think RPC frameworks are good for, is making you drop the convention of specific programming languages — or at least of whichever programming language they didn’t take after. Because clearly, various RPC frameworks inspire themselves from different starting languages, and so their conventions feel more or less at ease in each language depending on how far they are from the starting language.

So for instance, if you look at gRPC, errors are returned with a status code and a detailed status structure, while in Thrift you declare specific exception structures that your interfaces can throw. Both options make different compromise, and they require different amount of boilerplate code to feel more at ease with different languages.

There are programming languages, particularly in the functional family (I’m looking at you, Erlang!) that don’t really “do” error checking — if you made a mistake somewhere, you expect that some type of error will be raise/thrown/returned, and everything else will fall behind it. So an RPC convention with a failure state and a (Adam Savage voice) “here’s your problem” long stack trace would fit them perfectly fine.

This would be equivalent of having HTTP only ever return error codes 400 and maybe 500 — client or server error, and that’s about it. You deal with it, after all it’s nearly always a human in front of a computer looking at the error message, no? Well…

Turns out that being specific to a point of what your error messages are can be very useful, particularly when interacting at a distance (either physical distance, or the distance of not knowing the code of whatever you’re talking to) — which is now HTTP 401 is used to trigger an authentication request on most browsers. If you wanted to go a further step, you could consider a 451 response as an automated trigger to re-request the same page from a VPN in a different country (particularly useful with GDPR-restricted news sources in the USA, nowadays).

Personally, I think this is the reason why the dream of thin client libraries, in my experience, stays a dream. While, yes, with a perfectly specified RPC interface definition you could just use the RPC functions as if they were a library themselves… that usually means that the calls don’t “feel” correct for the language, for any language.

Instead, I personally think you need a wrapper library that can expose the RPC interfaces with a language-native approach — think builder paradigms in Java, and context managers in Python. Not doing so leads, in my experience, to either people implementing their own wrapper libraries you have no control over, or pretty bad code overall, because the people knowing the language refuse to touch the unwrapped client.

This is also, increasingly, relevant for local tooling — because honestly I’d rather have an RPC-based interface over Unix Domain Sockets (which allow you to pass authentication information) rather than running command line tools as subprocesses and trying to parse their output. And while for simpler services, signal-based communication or very simple “text” protocols would work just as well, there’s value in having a “lingua franca” to speak between different services.

I guess what I’m saying is that, unlike programming languages, I do think we should make, and stick to, choices on RPC systems. The fact that for the longest time most of Windows apps could share the same basic IPC/RPC system was a significant advantage (nowadays there’s… somewhat confusion at least in my eyes — and that probably has something to do with the amount of localhost-only HTTP servers that are running on my machines).

In the Open Source world, it feels like we don’t really seem to like the idea of taking options away – which was clearly visible when the whole systemd integration started – and that makes choices, and integrations, much harder. Unfortunately, that also means significantly higher cost to integrate components together — and a big blame game when some of the bigger, not-open players decide to make some of those choices (cough application-specific passwords cough).

Revisiting Open Source Washing Machines (in Theory)

A number of years ago I wrote a post about the idea of Free Software washing machines, using it as an allegory to point out how silly it might be to attack people for using products with closed-source firmware, when no alternative is readily available. At the time, the idea of wanting an open source washing machine was fairly ludicrous, despite me pointing out that it would have some interesting side effect with programmability.

Well, in 2020 I would actually suggest that I really wish we had more Open Source washing machines, and actually wish we had, in general, more hackable home appliances (or “white goods” as they are called over here), particularly nowadays that it seems like connected appliances are both trendy and the butt of many jokes.

It’s not just because having access to the firmware of everything is an interesting exercise, but because most appliances are lasting for years — which means they are “left behind” by technology moving on, including user interfaces.

Let’s take for instance washing machines — despite being older than the one in my apartment, my mother’s LG washing machine has a significantly more modern user interface. The Zanussi one we have here in London has one big knob to select the program – which is mostly vestigial from the time it would be spring-loaded and moving to select the various phases – and then a ton of buttons to select things like drying mode (it’s a washer/dryer combo), drying time, and the delay start (awesome feature). You can tell that the buttons were addition to an interface, and that the knob is designed to be as similar to the previous interface as possible. And turns out the buttons are not handy: both drying time and delay have only one button each — which means you can only increase those values: if you miss your target, you need to go back to zero and up again.

On the other hand, my mother’s LG also has a knob — but the knob is just a free-spinning rotary encoder connected to a digital controller. While her model is not a dryer, I’m reasonably sure that the machine has a delay start feature, which is configured by pressing one button and then rotating the wheel. A more flexible interface, with a display a bit more flexible than the two multi-segments that our current machine has, would do wonder to usability, and that’s without going into any of the possible features of a “connected appliance”. Observe-only, that is — I would still love seeing a notification on our phones when the washing machine completed, so that we don’t forget that we have clean clothes that need to be hanged to dry. Yes we actually forget sometimes, particularly before the pandemic if we left them to delay-run from the morning.

Replacing a washing machine just because the user interface is bad is a horrible thing to do for the planet. And in particular when living in rented accommodation, you own the white goods, and even when they are defective, you don’t get to choose them — you end up most of the time with whichever is the cheapest one in the shop, power efficiency be damned, since rarely the landlords are paying for electricity. So having hackable, modular washing machines would be just awesome: I could ask our landlord “Hey can I get a smartmodule installed for the washing machine? I’ll pay the £50 it costs!” (seriously, if it costs more than that, it would be a rip-off — most of the controls you need for this can be hardly more complicated than a Feather M4!)

Oh yeah and even if I just had access to the firmware of this washer/dryer I might be able to fix the bug where the “program finished” beeper does not wait for the door’s lock magnet to disengage before starting. The amount of times I need to set a timer to remind myself to go and take the towels out in five minutes is annoying as heck.

But it’s not just washing machines that would be awesome to be hackable and programmable. We have a smallish microwave and convection oven combo. I got it in Dublin, and I chose this model because it was recommended by an acquaintance for its insistent beeping when the timer completes. If you have ever experience hyperfocus at any degree, you probably understand why such a feature is useful.

But in addition to the useful feature, the oven comes with a bunch of pretty much useless ones. There’s a number of “pre-programmed” options for defrosting, or making pop-corns and other things like that, that we would never use. Not just because we don’t eat them, but also because they are rarely recommended — if you ever watch cooking channels such as How To Cook That, you find that professionals usually suggest specific way to use the microwave — including Ann Reardon’s “signature” «put it in the microwave for 30 seconds, stir, put it back for 30 seconds, stir, …».

And again in term of user interfaces, the way you configure the convection temperature is by clicking the “Convection” button to go from full power (200W) down — and if you got it wrong, oops! And then if you turn the knob (this time a free-spinning one, at least), you’re setting the timer, without pre-heating. If you want to pre-heat you need to cancel it all, and resume the thing, and… you see my point.

This is a very simple appliance, and it works perfectly fine. But if I could just open it and replace the control panel with something nicer, I would love to. I think that I would like to have something I can connect to a computer (or maybe connect an USB thumbdrive to), and configure my own saved parameters, selecting for instance “fish fingers” and “melted butter”, which are the more likely uses of the oven for us at home.

But again, this would require a significant change in the design of appliances, which I don’t think is going to happen any year now. It would be lovely, and I think that there might be a chance for Open Source and Open Hardware amateurs to at least show the possibility for it — but it’s the kind of project that I can only with for, with no hope to get to work on myself, not just for the lack of time but for the lack of space — if you wanted to try hacking on a washing machine, you definitely need a more spacious and populated workshop. My limit is still acrylic lamps.

Why I Care About Licensing

Both on the blog and on Twitter, I have ranted at length at projects missing licensing information altogether, or not providing licensing information on specific files, or providing conflicting licensing information. As you can imagine, this is a topic that I’m very attached to, which is why I have been following REUSE guidelines to make sure that all my (currently active) projects follow the specification.

Unfortunately this care is not shared with many developers, even those who consider themselves part of the Free Software movement, and this causes friction, poisons the well in both directions, and overall is detrimental to the community and the movement. Even more so than when people care deeply and disagree on the “correct” licensing terms.

While I am most definitely not a lawyer, and I speak most definitely only for myself and not my employer, let me try to give you a run down of what’s going on here.

First of all, we need to start with a simplification, and handwavey accept that without an explicit license allowing it, the distribution, modification, and integration of source code is not allowed, or at least that’s the way we perceive it in the wider world. And Free Software licenses, more or less permissive, spell out the terms with which distribution and usage are allowed.

It Is But An Example

As far as I can tell, there’s no provision anywhere that source code used in documentation is exempt from these limitations, except insofar as the license on the documentation itself would apply if not otherwise overridden. And that’s how I started engaging with Adafruit: the documentation for most of their CircuitPython libraries provide a lot of useful examples — and as it turns out they were already released with an open-source license (MIT), but that was not obvious when looking at the docs sites themselves. So I convinced them to add SPDX headers to all their source code, including the examples — and now you can read the example and see immediately which license it’s released under. Isn’t that cool?

Unfortunately, sometimes developers are stubborn and find adding two lines to their documentation examples a distraction, and argue against it, making it annoying for others to use their example source code without either infringing the copyright or going the long way to find the right answers.

Websites, PDFs, Books, they are all equal

But this goes to the double for code that is explicitly written only as example material! Let me take a bit of a detour — my wife went through the awesome Python Crash Course a few months ago. While it suffers from a few of the issues I already complained about when it comes to splitting names, the book is fairly well written and has hands-on exercise that provide enough of a stretch to “my first Python project”. In the later parts of the book, one of the long-building exercise is writing a clone of Space Invaders with PyGame, which turned out to be interesting not just for her writing it, but for myself reviewing it as well, as game programming is definitely not a skill I ever spent time acquiring.

Now, remember I said there’s space to stretch? While the book guides you through building the very basic framework for “Alien Invasion” with full code to go with it, it leaves a lot of holes to be filled. Not just the assets (that it pretty much suggests you Google for and find somewhere online, without any discussion on what you can and cannot use — shout out to the Noun Project which I use for my own projects nowadays), but also some of the more advanced gameplay, and a lot of the refactoring — the way you write the game following the book is definitely more aimed at teaching than at maintaining. So when my wife finished with the book, I started showing her examples of how to refactor the code and introduce new features. So while the basic skeleton is the same as the original from the book, the version she ended up with was nearly fully rewritten. And it’s all in a Git repository!

But she has nothing to show for it. The source code in the book does not provide any licensing information. When I reached out to Eric Matthes (the book’s author) on Twitter asking him if he’d consider applying an opensource license to the code, so that she could publish it on her GitHub account to show off to some of her friends – and with an explicit mention that I’d have liked to use it as a base to test out BeeWare projects and see to contribute to some – he said he’d think about it, but that he wouldn’t feel right to release it under a permissive license that would allow someone to take it and sell it on AppStore and similar. So her options are to ignore licensing and publish the code anyway (after all, nobody cares, and I’m sure I can find plenty of people who did exactly that), or to comply with the (lack of) license and keep it for herself, and only show her friends a video of it working. She went for the latter, as we already had a long discussion of copyright when J Salmeron brought up the topic (and dang, we missed the opportunity to shake his hand as we were standing right behind him at the Beast in Black concert in Amsterdam last year!)

Provide It And They Will Build

There is one case that, personally, drained my will to contribute to an ecosystem even more than the example above. After all, Python Crash Course is a great book, and the only really good reason to publish the code is for “bragging rights” — which is not to say it’s not something, but it’s not the end of the world either.

When a commercial vendor is providing you with an extensible ecosystem for you to build upon, but doesn’t play by the same rules, it’s just… disappointing. In this case the issue is with Saleae, the manufacturer of the Logic Pro 16 analyzer I use for a bunch of different things. You may have noticed me providing screenshots off it when talking about fake candles and infrared. As a vendor, Saleae has very good user support: when I complained on Twitter that I wasted two hours chasing ghosts because I didn’t realise I forgot to connect the USB cable to the analyzer, and the software didn’t make it clear enough it was showing me demo garbage, they engaged, asked me what I would have done differently, and delivered the fix in less than a month. That was awesome support.

So where does it go wrong? Well, in June they updated their software to support Python-based extensions for analysis of specific protocols. I was actually interested in adding support for IR decoding to make my life easier in my TV controlling project, and so when they posted that one of their employees built a duty cycle measure tool and posted it on GitHub I was thrilled!

Except… the repository is there, the source code is there, but there is no license. The extension is pretty much a tutorial by itself on how to build what I needed, but it’s coming with no license attached, and as such I can’t use its code as a base for my own extension. And while I could possibly learn from it, it’s also a poison pill… there’s no license, if I copy it too literally, am I infringing copyright? Maybe, who knows? The author says I should «feel free to look, copy and use [his] Logic 2 extensions in any way [I] would like», but that’s not exactly a very comforting statement when you’re contributing while part of a company.

Final Thoughts

~~Just be yourself (this is pre-recorded).~~ If you do care about Free Software, please take licensing seriously. If you don’t care about Free Software, because you don’t believe in the ideals behind, or you’re just not part of the ecosystem, then I can’t really blame you for disrespecting licenses, but then again if you rely on proprietary software license, you probably should respect all of them. It’s the same problem with software piracy.

I do believe that the folks at REUSE are doing a great service for all of us by making it possible to spell out licenses clearly and openly, and making it easy for others to modify and copy the code that we want to be out there in the world. It doesn’t take so much time to use the tool to add a few lines to a text file, or an additional text file for binary files. Please take the chance to sort this out!

Converting Unpaper to Meson

You may remember that I took over Unpaper a number of years ago, as the original author was not interested in maintaining it in the long term. And one of the first things I did was replacing the hand-rolled build system with Autotools, as that made packaging it significantly simpler. Followed by replacing the image decoding with libav first, and more recently FFmpeg.

For various reasons, I have not spent much time on Unpaper over the years I spent in my last bubble. When I joined my current employer I decided that I cared more to get Unpaper back into a project people can contribute to, and less about maintaining the copyright of my contributions, which makes it easier for me to work on it without having to carve out some special time for it.

One of the main requests over on the GitHub project, over these years, has been Windows support. And I did say that Windows 10 is becoming increasingly interesting for Free Software development. So when I was asked to rant on a bit about build systems, which I did over on Twitch, I decided to take a stab at figuring out if Meson (which supports Windows natively), would be a good fit. And I did that using Visual Studio Code and Debian/WSL!

If you haven’t seen the video yet, spoiler alert: I tried, got it working within ten minutes, and it worked like a charm. I’m not kidding you, it pretty much worked at the first attempt (well, the first session, not the first execution), and it made total sense the way it works. You can tell that the folks involved in building Meson (including Gentoo’s own Nirbheek!) knew what they were embarking to do and how to have it fit together. Even small bits like keeping large files support always enabled made me a very happy user.

I have now a branch on GitHub for the Meson build, although it’s incomplete. It doesn’t install any of the docs, and it doesn’t install the man page. Also it doesn’t build nor run the tests. For all of those I created a project to track what needs to be done: move on from the current implementation, it’s 2020!

The test system in the current Autotools version of Unpaper is leveraging the implementation of make check from Automake, together with a C program that compares the expected outputs with what is generated by the newly-built unpaper binary. It also needs to consider a threshold of difference between the two, because precision is not guaranteed, in either direction. This is incredibly fragile and indeed is currently failing for… not sure which reason. Getting a “diff” of the generated versus expected in C is fairly hard and deserves its own project. Instead, relying on Python for instrumenting and running the tests would make it much easier to maintain, as you wouldn’t need to maintain at least three integration points to keep this together.

Something similar is a problem for documentation: right now the documentation is split between some Markdown files, and a single DocBook (XML) file that is converted to a man page with xsltproc. Once the bullet of Python is bit, there’s no reason not to just use ReStructuredText and Sphinx, which already provides integration to generate man pages — and honestly nowadays I feel like the XML version is definitely not a good source because it can’t be read by itself.

What I have not done yet is making sure that the Meson build system allows a Windows build of Unpaper. The reason is relatively simple: while I have started using Visual Studio Code, and clang is available for Windows in the form of a binary, and so is FFmpeg, fitting it all together will probably take me more time, as I’m not used to this development environment. If someone is interested in making sure this works out of the box, I’m definitely happy to review pull requests.

So yeah, I guess the Autotools Mythbuster can provide a seal(ion) of approval to Meson. Great work, folks! Happy to see we have a modern build system available that compromises in the right places instead of being too dogmatic!

The Rolodex Paradigm

Silhouette of a rolodex.

Created by Marie-Pierre Bauduin from Noun Project.

In my previous bubble, I used to use as my “official” avatar a clipart picture of a Rolodex. Which confused a lot of people, because cultures differ and most importantly generation differ, and turned out that a lot of my colleagues and teammates never had seen or heard of a Rolodex. To quote one of the managers of my peer team when my avatar was gigantic on the conference room’s monitor «You cannot say that you don’t know what a Rolodex is, anymore!»

So, what is a Rolodex? Fundamentally, it’s a fancy address book. Think of it as a physical HyperCard. As Wikipedia points out, though, the name is sometimes used «as a metonym for the sum total of an individual’s accumulated business contacts», which is how I’m usually using it — the avatar is intentionally tongue-in-cheek. Do note that this is most definitely not the same as a Pokédex.

And what I call the Rolodex Paradigm is mainly the idea that the best way to write software is not to know everything about everything, but to know who knows something about what you need. This is easier said than done of course, but let me try to illustrate why I mean all of this.

One of the things I always known about myself is that I’m mainly a generalist. I like knowing a little bit about a lot of things, rather than a lot about a few things. Which is why on this blog you’ll find superficial posts about fintech, electronics, the environment, and cloud computing. You’ll rarely find in-depth articles about anything more recently because to get into that level of details I would need to get myself “in the zone” and that is hardly achievable while maintaining work and family life.

So what do I do when I need information I don’t have? I ask. And to do that, I try to keep in mind who knows something about the stuff that interest me. It’s the main reason why I used to use IRC heavily (I’m still around but not active at all), the main reason why I got to identi.ca, the main reason why I follow blogs and write this very blog, and the main reason why I’m on social networks including Twitter and Facebook – although I’ve switched from using my personal profile to maintaining a blog’s page – and have been fairly open with providing my email address to people, because to be able to ask, you need to make yourself available to answer.

This translates similarly in the workplace: when working at bigger companies that come with their own bubble, it’s very hard to know everything of everything, so by comparison it can be easier to build up a network of contacts who work on different areas within the company, and in particular, not just in engineering. And in a big company it even has a different set of problems to overcome, compared to the outside, open source world.

When asking for help to someone in the open source world, you need to remember that nobody is working for you (unless you’re explicitly paying them, in which case it’s less about asking for help and more about hiring help), and that while it’s possible that you’re charismatic enough (or well known enough) to pull off convincing someone to dedicate significant amount of time to solve your problems, people are busy and they might have other priorities.

In a company setting, there’s still a bit of friction of asking someone to dedicate a significant amount of time to solve your problem rather than theirs. But, if the problem is still a problem for the company, it’s much more likely that you can find someone to at least consider putting your problem in their prioritised list, as long as they can show something for the work done. The recognition is important not just as a way to justify the time (which itself is enough of a reason), but also because in most big companies, your promotion depends on demonstrating impact in one way or another.

Even were more formal approaches to recognitions (such as Google’s Peer Bonus system) are not present, consider sending a message to the manager of whoever helped you. Highlight how they helped not just you personally, but the company — for instance, they may have dedicated one day to implement a feature in their system that saved you a week or two of work, either by implementing the same feature (without the expertise in the system) or working around it; or they might have agreed to get to join a sketched one hour meeting to provide insights into the historical business needs for a service, that will stop you from making a bad decision in a project. It will go a long way.

Of course another problem is to find the people who know about the stuff you need — particularly if they are outside of your organization, and outside of your role. I’m afraid to say that it got a lot harder nowadays, given that we’re now all working remote from different houses and with very little to no social overlapping. So this really relies significantly on two points: company culture, and manager support.

From the company point of view, letting employees built up their network is convenient. Which is why so many big companies provide spaces for, and foster, interaction between employees that have nothing to do with work itself. While game rooms and social interactions are often sold as “perks” to sell roles, they are pretty much relaxed “water cooler” moments, that build those all-too-precious networks that don’t fit into an org structure. And that’s why inclusive social events are important.

So yeah, striking conversations with virtual stranger employees, talking about common interests (photography? Lego? arts?) can lead into knowing what they are working on, and once they are no longer strangers, you would feel more inclined to ask for help later. The same goes for meeting colleagues at courses — I remember going to a negotiation training based around Stuart Diamond’s Getting More, and meeting one of the office’s administrative business partners, who’s also Italian and liking chocolate. When a few months later I was helping to organize VDD ’14, I asked her help to navigate the amount of paperwork required to get outsiders into the office over a weekend.

Meeting people is clearly not enough, though. Keeping in touch is also important, particularly in companies where teams and role are fairly flexible, and people may be working on very different projects after months or year. What I used to do for this was making sure to spend time with colleagues I knew from something other than my main project when traveling. I used to travel from Dublin to London a few times a year for events — and I ended up sitting close to teams I didn’t work with directly, which lead me to meeting a number of colleagues I wouldn’t have otherwise interacted with at all. And later on, when I moved to London, I actually worked with some of them in my same team!

And that’s where the manager support is critical. You won’t be very successful at growing a network if your manager, for example, does not let you clear your calendar of routine meetings for the one week you’re spending in a different office. And similarly, without a manager that supports you dedicating some more time for non-business-critical training (such as the negotiation training I’ve mentioned), you’ll end up with fewer opportunities to meet random colleagues.

I think this was probably the starkest difference between my previous employer’s offices in Dublin and London: my feeling was that the latter had far fewer opportunities to meet people outside of your primary role and cultivate those connections. But it might also be caused by the fact that many more people live far enough from the office that commuting takes longer.

How is this all going to be working in a future where so many of us are remote? I don’t honestly know. For me, the lack of time sitting at the coffee area talking about things with colleagues that I didn’t share a team with, is one of the main reason why I hope that one day, lockdown will be over. And for the rest, I’m trying to get used to talk over the Internet more.

Birch Books With ATmega48

I last wrote about the Trinket M0 version of Birch Books, which is using CircuitPython and uses a much simplified code structure, while relying on one of two possible I/O expanders. But I felt a bit of a cheat to implement this one with a Trinket M0, which is a very powerful MCU to use for something that should be as simple as to keep a few lights on in a LEGO set.

So, as suggested by a colleague, I looked into the ATmega48, which is a close relative to the ATmega328 that Arduino is based on. Actually, these two versions are pin- and source-compatible so while I have now implemented it with the 48, the 328 (which is more commonly available) should work the same way.

The ATmega48 option is an interesting compromise between the 8052 and the Trinket M0. It’s an 8-bit microcontroller, and it requires to write low-level C to program, but at the same time it doesn’t require any passive components (as long as you don’t need more than 8MHz, and you didn’t make the mistake I made about having an active-high RST button). It turned out that the suggestion was a very good one, and the ATmega48 does indeed seem to fit this project very nicely.

Programming the ATmega48 can be done via 6-pin connectors that need to provide power, SPI bus, and the RST line. This can be achieved with Adafruit’s FT232H board: you connect the SPI lines just as described on the breakout, and use D3 for the RST line, then you just tell avrdude to use the UM232H variant. No other configuration is needed. I actually plan on crimping my own harness for this later, so I don’t need to use a breadboard all the time.

The other main difference between the ATmega48 and the 8052 is that the former has nearly half the pins, but it still has enough I/Os (14 outputs, 2 inputs) for Birch Books that there is no need to add an expander. There’s a few more I/O lines as well, but they are “overcommited” and by default configured to have different meanings. For instance the external clock pins can be re-configured to be additional I/O lines, and even the RST line can be disabled by fuses to gain one more I/O.

Let’s talk about the clock! The ATmega48 is sent out from factory with the fuses configured for a 1MHz clock, but can be configured up to 8MHz. Alternatively it can use an external clock and run up to… I forgot, and the point being I don’t need any of that. I did order 100 pieces of 12MHz crystals just to have them at hand, when I made an order from LCSC, but for this project, the 1MHz clock speed is plenty. So I left it configured the way it came from factory, and that works perfectly fine.

The code ended up being very similar to the 8052 version of course: it initializes the ports, sets up a timer input, and in the loop checks for the test button and otherwise keep updating the port state depending on the calculated virtual hour. While the ATmega48 would have allowed for more fine-grained controls on the timing, and thus allowed me to use the original planned scheduled, I decided to actually keep the same 1/16th of a second “tick” timer, and the same formulas to calculate the schedule.

Another thing allowed by the ATmega48, or rather the fact by the AVR-GCC, would be making the code more modular. As I ranted before, SDCC is not the most aggressive compiler, when it comes to optimizing around functions, and that’s why instead of having more structured code I ended up with a lot of spaghetti. To make it easier to compare and maintain the code together, I decided not to refactor the spaghetti in the AVR version, but I think that’s okay. The simplification around the timer handling is already a huge step forward anyway.

On the logic side, I decided to implement a one second power-on self-test though that turns on all of the lights. This was meant not just to easily test the code while I had it on my desk, but also just as a general “healthiness check” for the LEDs. I backported it to the CircuitPython code while I was at it, as well. And I removed the “knuckle pattern strobing” (which turned on every other LED alternatively), because it doesn’t work quite right or well on the real LEGO set (where two rooms use three LED lines tied together), simplifying the test logic.

The other thing that I found out by using the ATmega48 is that I do have some difference in the current limiting between the Trinket M0 and the other two controllers. When I installed the Trinket M0-based controller, the lights in the bookstore were much brighter, and even if that looked nice, it had me worried, because I knew that the LEDs I’ve been using tend to brown out quickly if they are powered at maximum current. I managed to pretty much kill one LED by keeping it running for just two days without a current-limiting resistor. And indeed, a month later, some of the LEDs are now nearly completely dark, and will need to be replaced.

Since the actuator board is the same, powered from 5V, and with 47 Ohm resistor networks for the LEDs in each case, I’m not entirely sure why this is happening. I might need to add some inline current metering to to see how that fits together. But that’s for another time.

Newcomers, Teachers, and Professionals

You may remember I had already a go at tutorials, after listening in on one that my wife had been going through. Well, she’s now learning about C after hearing me moan about higher and lower level languages, and she did that by starting with Harvard’s CS50 class, which is free to “attend” on edX. I am famously not a big fan of academia, but I didn’t think it would make my blood boil as much as it did.

I know that it’s easy to rant and moan about something that I’m not doing myself. After all you could say “Well, they are teaching at Harvard, you are just ranting on a c-list blog that is followed by less than a hundred people!” and you would be right. But at the same time, I have over a decade of experience in the industry, and my rants are explicitly contrasting what they say in the course to what “we” do, whether it is in opensource projects, or a bubble.

I think the first time I found myself boiling and went onto my soapbox was when the teacher said that the right “design” (they keep calling it design, although I would argue it’s style) for a single-source file program is to have includes, followed by the declaration of all the functions, followed by main(), followed by the definition of all the functions. Which is not something I’ve ever seen happening in my experience — because it doesn’t really make much sense: duplicating declarations/definitions in C is an unfortunate chore due to headers, but why forcing even more of that in the same source file?

Indeed, one of my “pre-canned comments” in reviews at my previous employer was a long-form of “Define your convenience functions before calling them. I don’t want to have to jump around to see what your doodle_do() function does.” Now it is true that in 2020 we have the technology (VSCode’s “show definition” curtain is one of the most magical tools I can think of), but if you’re anyone like me, you may even sometimes print out the source code to read it, and having it flow in natural order helps.

But that was just the beginning. Some time later as I dropped by to see how things were going I saw a strange string type throughout the code — turns out that they have a special header that they (later) define as “training wheels” that includes typedef char *string; — possibly understandable given that it takes some time to get to arrays, pointers, and from there to character arrays, but… could it have been called something else than string, given the all-too-similarly named std::string of C++?

Then I made the mistake of listening in on more of that lesson, and that just had me blow a fuse. The lesson takes a detour to try to explain ASCII — the fact that characters are just numbers that are looked up in a table, and that the table is typically 8-bit, with no mention of Unicode. Yes I understand Unicode is complicated and UTF-8 and other variable-length encodings will definitely give a headache to a newcomer who has not seen programming languages before. But it’s also 2020 and it might be a good idea to at least put out the idea that there’s such a thing as variable-length encoded text and that no, 8-bit characters are not enough to represent people’s names! The fact that my own name has a special character might have something to do with this, of course.

It went worse. The teacher decided to show some upper-case/lower-case trickery on strings to show how that works, and explained how you add or subtract 32 to go from one case to the other. Which is limited not only by character set, but most importantly by locale — oops, I guess the teacher never heard of the Turkish Four Is, or maybe there’s some lack of cultural diversity in the writing room for these courses. I went on a rant on Twitter over this, but let me reiterate this here as it’s important: there’s no reason why a newcomer to any programming language should know about adding/subtracting 32 to 7-bit ASCII characters to change their case, because it is not something you want to do outside of very tiny corner cases. It’s not safe in some languages. It’s not safe with characters outside the 7-bit safe Latin alphabet. It is rarely the correct thing to do. The standard library of any programming language has locale-aware functions to uppercase or lowercase a string, and that’s what you need to know!

Today (at the time of writing) she got to allocations, and I literally heard the teacher going for malloc(sizeof(int)*10). Even to start with a bad example and improve from that — why on Earth do they even bother teaching malloc() first, instead of calloc() is beyond my understanding. But what do I know, it’s not like I spent a whole lot of time fixing these mistakes in real software twelve years ago. I will avoid complaining too much about the teacher suggesting that the behaviour of malloc() was decided by the clang authors.

Since there might be newcomers reading this and being a bit lost of why I’m complaining about this — calloc() is a (mostly) safer alternative to allocate an array of elements, as it takes two parameters: the size of a single element and the number of elements that you want to allocate. Using this interface means it’s no longer possible to have an integer overflow when calculating the size, which reduces security risks. In addition, it zeroes out the memory, rather than leaving it uninitialized. While this means there is a performance cost, if you’re a newcomer to the language and just about learning it, you should err on the side of caution and use calloc() rather than malloc().

Next up there’s my facepalm on the explanation of memory layout — be prepared, because this is the same teacher who in a previous lesson said that the integer variable’s address might vary but for his explanation can be asserted to be 0x123, completely ignoring the whole concept of alignment. To explain “by value” function calls, they decide to digress again, this time explaining heap and stack, and they describe a linear memory layout, where the code of the program is followed by the globals and then the heap, with the stack at the bottom growing up. Which might have been true in the ’80s, but hasn’t been true in a long while.

Memory layout is not simple. If you want to explain a realistic memory layout you would have to cover the differences between physical and virtual memory, memory pages and pages tables, hugepages, page permissions, W^X, Copy-on-Write, ASLR, … So I get it that the teacher might want to simplify and skip over a number of these details and give a simplified view of how to understand the memory layout. But as a professional in the industry for so long I would appreciate if they’d be upfront with the “By the way, this is an oversimplification, reality is very different.” Oh, and by the way, stack grows down on x86/x86-64.

This brings me to another interesting… mess in my opinion. The course comes with some very solid tools: a sandbox environment already primed for the course, an instance of AWS Cloud9 IDE with the libraries already installed, a fairly recent version of clang… but then decides to stick to this dubious old style of C, with strcpy() and strcmp() and no reference to more modern, safer options — nevermind that glibc still refuses to implement C11 Annex K safe string functions.

But then they decide to not only briefly show the newcomers how to use Valgrind, of all things. They even show them how to use a custom post-processor for Valgrind’s report output, because it’s otherwise hard to read. For a course using clang, that can rely on tools such as ASAN and MSAN to report the same information in more concise way.

I find this contrast particularly gruesome — the teacher appears to think that memory leaks are an important defect to avoid in software, so much so that they decide to give a power tool such as Valgrind to a class of newcomers… but they don’t find Unicode and correctness in names representation (because of course they talk about names) to be as important. I find these priorities totally inappropriate in 2020.

Don’t get me wrong: I understand that writing a good programming course is hard, and that professors and teachers have a hard job in front of them when it comes to explain complex concepts to a number of people that are more eager to “make” something than to learn how it works. But I do wonder if sitting a dozen professionals through these lessons wouldn’t make for a better course overall.

«He who can, does; he who cannot teaches» is a phrase attributed to George Bernand Shaw — I don’t really agree with it as it is, because I met awesome professors and teachers. I already mentioned my Systems’ teacher, who I’m told retired just a couple of months ago. But in this case I can tell you that I wouldn’t want to have to review the code (or documentation) written by that particular teacher, as I’d have a hard time keeping to constructive comments after so many facepalms.

It’s a disservice to newcomers that this is what they are taught. And it’s the professionals like me that are causing this by (clearly) not pushing back enough on Academia to be more practical, or building better courseware for teachers to rely on. But again, I rant on a C-list blog, not teach at Harvard.