Glucometer Notes: GlucoRx Q

This article is going to be spending some time to talk about the meter, the manufacturer, and my reverse engineering. The more “proper” review of the device will be at the end, look for Review as title.

So despite having had no progress in months with my reverse engineering efforts started last year, I have not been ignoring my pastime of acquiring and reverse engineering the protocols of glucometers. And since I felt a bit bored, I went onto ~~AliExpress~~ Amazon UK and searched for something new to look at. Unfortunately, as usual, Amazon is becoming more and more a front for drop-ship sellers from AliExpress and similar sources, so most of the results are Sinocare. Eventually, I found the a GlucoRx Q, which looked interesting, given that I have already reverse engineered the GlucoRx Nexus.

Let’s start with a few words about the brand, because that’s one of those “not-so-secret secrets” that is always fun to remind people of: GlucoRx does not actually design, or manufacture, these devices or the strips they use. Instead, they “rebadge” white label glucometers. The Nexus meter I previously looked at was also marketed by the Italian company Menarini and the German (if I understand that correctly) Aktivmed, but was actually manufactured by TaiDoc, a Taiwanese company, as the TD-4277. I say this is not so secret because… it’s not a secret. The name of TaiDoc, and the original model number are printed on the label at the bottom of the device.

Now, some manufacturers doing this kind of white label rebadging don’t really have “loyalty” to a single manufacturer, so when I saw that the Q required different strips from the Nexus, I thought it would be a different manufacturer this time, which brought up my hopes that I would have a fun reverse engineering project on my hands, but that turned out to be disappointed very quickly, as the device said it’s a TaiDoc TD-4235B.

A quick search on Wikidata turned out to be even more interesting than I expected, and showed that GlucoRx markets more of the TaiDoc devices too, including the Nexus Voice TD-4280. Interesting that the company does not have an entity at the time of writing, and that even the retracted article names TaiDoc twice, but GlucoRx 45 times. To make a comparison with computers, it’s like writing an article about a Surface Book laptop and keep talking about the CPU as if it was Microsoft’s.

Anyway, even though the manufacturer was the same, I was still hoping to have some fun reverse engineering it. That was also disappointed: it took me longer to set up Windows 10 in a virtual machine than it took me to make glucometerutils download the data from the meter. It looks like TaiDoc has a fairly stable protocol, which honestly surprised me, as a lot of the manufacturers appear to just try to make it harder to support their devices.

Indeed this meter also shows up with a CP2110-compatible HID endpoint, which meant I could reuse my already-written chatter script to extract the back-and-forth between the Windows app and the device, and confirm that it was pretty much the same as the Nexus. The only differences were the model number (which is still issued in little-endian BCD), and a couple of unknown bytes that weren’t as constants as I thought they were. I also updated the documentation.

Why did I say “CP2110-compatible” instead of just CP2110? Well, here’s the thing: the GlucoRx Q shows up on the kernel logs (and in Windows hardware notifications) as “Silicon Laboratories C8051F34x Development Board”. Sounds like someone forgot to flash in the magic strings, and that pretty much “broke the magic” of which platform these devices are based on. Again, not the biggest secret, but it’s always interesting.

As the name might already have given away, the Silicon Labs C8051F34x is an 8-bit microcontroller based on the 8051. Yes, the same architecture I used for Birch Books, and for which I complained about the lack of good FLOSS support (since there doesn’t seem to be any institutional money to improve). It appears that these MCUs don’t just include the 8051 core but also a whole suite of components that do make them very versatile for the use on glucometers, namely fast and precise Analog-to-Digital Converters (ADCs). It also appears to have an integrated USB-to-serial through the same HID protocol as the CP2110.

So, yeah, I’m considering doing one run of the Birch Books controller based on this particular MCU out of curiosity, because they come in a package that is still hand-solderable and include already USB support. But that’s a project for another time.

So putting the reverse engineering (or rather, the no lack of need of it) aside, let’s take a quick look at the meter as a meter.

Review

There is not much to say about this meter, because it’s your average “cheap” meter that you can find on online stores and pharmacies. I’m still surprised that most people don’t just get a free meter from one of the big names (in Italy, Ireland, and UK they are usually free — the manufacturers make their money on the strips), but this is not uncommon.

The GlucoRx Q is a fairly comfortable meter — unlike the Nexus, it’s “pill-shaped”, reminding me a lot of the Contour Next One. Unlike the Contour, this meter is not backlit, which means it’s not usable in dark places, but it also has a significantly bigger display.

The size does not compare entirely favourably with the FreeStyle Libre, part of the reason for which is that it runs off a single AAA battery — which makes it easy to replace, but puts some constraints on the size. On the bright side, the compartment door is captive to the main body so you don’t risk losing it if you were to change the battery on a moving vehicle, for instance.

The fitting of the strips is nice and solid, but I have to say getting blood onto them was quite harder than other meters, including the already mentioned Sinocare. Unlike other meters, there’s no lever to eject the strip without touching it — which makes me wonder once again if it’s a cultural reason for most of the Chinese meters to have it.

As usual for these reviews, I can’t really give an opinion on the accuracy — despite having spent many years looking at glucometers, I haven’t figured out a general way test these for accuracy. Most meters appear to have a calibration solution, but that’s not mean tot be compatible between devices, so I have no way to compare them to each other.

I don’t see any particular reason for getting or avoiding this particular device, to be honest. It seems to just be working fine, but at the same time I get other meters for free, and the strips are covered by NHS for me and all the diabetics — if anyone has any other reason on why to prefer this meter, I’d love to hear about it.

Service Announcement: Pawsome Players Streaming Week

You may remember I have been irregularly streaming on Twitch some of my FLOSS work, which focused almost exclusively on unpaper over the past few months. Now, unrelated to this, Cats Protection, the British no-profit whose primary goal is to rehome cats in need, launched their Pawsome Players initiative, aiming at raising fund through streaming — video games primarily, but not just.

With this in mind, I decided to join the fun, and will be streaming for the whole week at least an hour every day, and work on more FLOSS projects. You can find the full schedule (as well as donate to the campaign) on Tiltify, and if you want to get reminded of a particular night, you can join the events on the blog’s Facebook page.

In addition to wrapping up the Meson conversion of Unpaper, I’m planning to do a little bit of work on quite a few more other projects:

I have a new glucometer I want to reverse engineer, and with that comes an opportunity to see my way of working through this type of work; I’m not as entertaining and deep as Hector, but if you have never looked over the shoulder of a “black box” reverse engineer, I think it might be interesting. The code I’ll be working on is likely usbmon-tools rather than glucometerutils, but there’s a chance that I’ll get so far ahead I’ll actually implement the code.
Speaking of reverse engineering, I have a few adapters I designed (and got printed) for my Saleae Logic Pro 16. I have not released the schematics for those yet, but I now have the work approvals to. I should make a repository for them and release them, I’ll do that on stream!
I want to make some design changes to my Birch Books, which I’ll discuss on stream. It’s going to be a bit more PCB “designing” (I use quotes here, because I’m far from a designer, and more of a “cobbler together”) which is likely going to be scary for those who do know what they are doing.

I’m also open to the idea of doing some live code-reviews — I did lots of those when working at Google, and while for those I had a lot of specific standards to appeal to, a number of suggestions are nearly universal, and I have done this before where I was pointed at some Python project and gave some general advice of what I see. I’d be willing to take a random project and see what I can notice, if the author is willing!

Also, bonus points for anyone who guesses where the name of the fundraising page from.

So I hope I’ll hear from you on the stream chat (over on Twitch), and that you’ll join me in supporting Cats Protection to help find every kitty a forever home (me and my wife would love to be one of those homes, but it’s not easy when renting in London), and reach the £1985 target.

Senior Engineering: Give Them Time To Find Their Own Stretch

This blog post is brought to you by a random dream I had in December.

As I reflect more on what it means for me to be considered a senior engineer, I find myself picking up answers from my own dreams as well. In a dream this past December, I felt myself being at the office (which, given the situation, is definitely a dream state) but also in particular I found that I was hosting a number of people from my old high school.

As it turns out it’s well possible that the reason why I dreamt of my old high school is because I was talking with my wife about how I would like there to be an organization that would reach out in schools such as that one. But that’s not what the dream was about. And it probably was helped along by having seen a Boston Legal episode about high schools the night before.

In the dream, I was being asked by one of the visitors what would I would recommend them to do, so that more of their students would be successful, and work at big companies. Small aside: there were three of us from my high school, and my year, working in Google when I joined – as I understand one of them was in New York and then left, but I was never really in touch with him – the other guy was actually in Dublin the same as me for a while, and I knew him well enough that I visited him when he moved away.

What I answered was something along the lines of: just don’t fill them of homework that takes them time. They won’t need the homework. They need to find something they can run with. Give them time to find their own stretch.

This is likely part of my personal, subjective experience. I generally hated homework from the beginning, and ignored most of it most of the time, if I could get away with it (and sometimes even when I couldn’t). Except when the homework was something that I could build upon myself. So while on most of the days I would be coming home from school and jump on Ultima OnLine right after lunch, other times I would be toying with expanding on what my programming teacher gave me or, as I told already, with writing a bad 8086 emulator.

But it reminded me of the situation at work, as well — “stretch” being a common word used to refer to the work undertaken by tech employees outside of their comfort zone or level, to “reach” their next goal or level. This plays especially into big tech companies, where promotions are given to those who already act as the next level they’re meant to reach.

While thankfully I only experienced this particular problem first hand a few times, and navigated most of them away, this is not an uncommon situation. My previous employer uses quarterly OKR planning to set up the agenda for each quarter, and particularly in my organization, the resulting OKRs tended to over-promise, by overcommitting the whole team by design. That meant taking on enough hi-priority tasks (P0 as they are usually referred to) to take the whole time of many engineers just to virtually keep the lights on.

Most of this is a management problem to solve, particularly when it came to setting expectations with partner teams about the availability of the engineers who could be working on the projects, but there’s a number of things that senior engineers should do, in my opinion, to prevent this from burning junior engineers.

When there’s too many “vertical” stakeholders, I still subscribe to the concept of diagonal contributions – work on something that brings both you, and many if not all of your verticals, in a better place, even when you could bring a single vertical to a much better place ignoring all the others – but for that to work out well, you need to make the relationship explicit, having buy-in from the supported teams.

The other important point is just to make sure to “manage upwards”, as I’ve been repeatedly told to do in the previous bubble, making sure that, if you have juniors in your team, they are given the time for training, improving, collaborating. That means being able to steer your manager away from overcommitting those who need more time to ramp up. It’s not a hypothetical — with my last manager at my previous role, I had to explicitly say a few times “No, I’m not going to delegate this to Junior Person, because they are already swamped, let them breathe.”

As I said previously, I think that it’s part and parcel of a senior engineer job to let others do the work, and not being the one concentrating the effort — so how do you reconcile the two requirements? I’m not suggesting it’s easy, and it definitely varies situation by situation. In the particular case I was talking about, the team was extremely overcommitted, and while I could have left a task for Junior Person to work on for a quarter or maybe two, it would have gone to be an urgent task afterwards, as it was blocking a migration from another team. Since it would have taken me a fraction of the time, I found it more important for Junior Person to find something that inspired them, rather than something that our manager decided to assign.

Aside: a lot can be written about the dynamics of organizations where managers get to say “I know they said the deadline is Q4, but nobody believes that, we’ll work on it in Q1 next year.” The technical repercussion of this type of thinking is that migrations drag along a lot longer than intended and planned, and supporting the interim state makes them even more expensive on both the drivers and receivers of the migration itself. Personally, if I’m told that Q4 is a deadline, I always tried to work towards migrating as many of the simple cases by Q3. That allowed me identifying cases that wouldn’t be simple, making requests to the migration drivers if needed, and left Q4 available for the complex cases that needed a lot more work. If you don’t do that, you risk assuming that something is a simple case up until the deadline, and then find out it’s another quarter worth of project work to address it.

Of course, a lot of it comes down to incentives, as always. My understanding of the state of it before I left my previous company is that the engineers are not evaluated directly on their OKR completion, but managers are evaluated on their team’s. Making that clear and explicit was itself a significant life improvement I would say — particularly because this didn’t use to be the case, and I was not alone in having seen the completion rate as being a fantasy number that didn’t really do much — much has been written by others on OKR and planning, and I’m no expert at that, so I won’t go into details of how I think about this altogether.

Keeping this in mind, another factor in providing stretch to more junior engineers, is to provide them with goals that are flexible enough for them to run with something they found interesting — leaving your task as the senior person to make sure the direction is the right one. To give an example, this means that instead of suggesting one specific language for implementing a tool, you should leave it up to the engineer working on it to choose one — within the language choice the team has accept, clearly.

It also means not requiring specific solutions to problems, when the solution is not fully fleshed out already. Particularly when looking at problems that apply to heterogenous systems, with different teams maintaining components, it’s easy to fall into the trap of expecting the solution to be additive – write a new component, define a new abstraction layer, introduce a compatibility library – ignoring subtractive solutions — remove the feature that’s used once every two years, delete the old compatibility layer that is only used by one service that hasn’t been ported yet. Focusing on what the problem to be solved is, rather than how it needs to be solved, can give plenty of room to stretch.

At the very least this worked for me — one of the tasks I’ve been given in my first team was to migrate between two different implementations of a common provider. While I did do that to “stop the fire”, I also followed up by implementing a new abstraction on top of it, so I could stop copy-pasting the same the code in ten different applications. I maintained that abstraction until I left the company, and it went from something used within my team only, to the suggested solution to that class of problems.

The other part that I want to make sure people consider, is that deciding that what you were assigned is not important and that there’s better things to do, is not a bad thing. Trying to make it harder to point out “Hey, I was assigned (or picked up) this goal, but after talking with stakeholders directly there’s no need to work on it” is just going to lead to burnout.

To go back to the continuous example of my last team at the previous employer, I found myself having to justify myself for wanting to drop one of the goals from my OKR selection – complicated by the fact that OKR completion was put on my performance plan – despite the fact that the area expert already signed off on it. The reason? We were in a month already in the quarter, and it seemed to late in the game to change our mind. Except the particular goal was not the highest priority, and the first month of the quarter was pretty much dedicated full time to one that was. Once I started poking at the details of this lower priority goal, I realized it would not gain anything for anyone – it was trading technical debt in one place to technical debt in another – while we had a more compelling problem on a different project altogether.

Again, all of this does not lead to a cookie-cutter approach — even senior engineers joining a new space might need to be told what to do so that they are not just wasting air for a while. But if you’re selecting tasks for others to work on, make sure that they get an opportunity to extend on it. Don’t give them a fully-detailed “just write the code for this very detailed specification” if you can avoid it, but rather tell them “solve this problem, here’s the tools, feel free to ask for details”.

And make sure you are around to be asked questions, and to coach, and to show what they may not know. Again from personal experience, if someone comes to you saying that they’d like advice on how to write design docs that pass arbitrarily complicated review processes, the answer “Just write more of them” is not useful. Particularly when the process involves frustrating, time-consuming synchronous reviews with people wanting you to know the answer to all their “What if?” questions without having heard them before.

In short, if you want your juniors to grow and take your place – and in my opinion, you should want that – then you should make sure not to manage their time to the detail. Let them get passionate about something they had to fix, and own seeing it fixed.

Flashing ESPHome on White Label Smart Plugs

You may remember that I bought a smart plug a few years ago, for our Christmas tree, and had been wondering what the heck do you use a smart plug for. Well, a few months ago I found out another use case for it that I had not thought about to that point: the subwoofer.

When I moved to Dublin, nearly eight years ago now, I bought a new TV, and an AV receiver, and a set of speakers — and with these last one, came a subwoofer. I don’t actually like subwoofer that much, or at least I didn’t, because the low-frequency sounds give (or gave) me headaches, and so while it does sound majestic to watch a good movie with the subwoofer on, I rarely do that.

On the other hand, it was still preferable to certain noise coming from the neighbours, particularly if we ended up not being in the same room as the device itself. Which meant we started using the subwoofer more — but unlike the AV receiver, that can be controlled with a network endpoint, the subwoofer has a physical switch to turn on and off… a perfect use case for setting this up with a smart plug!

Since I didn’t want to reuse the original plug for the Christmas tree (among other things, it’s stashed away with the Christmas tree), I ended up going online and ordering a new one — and I selected a TP-Link, that I thought would work just the same as my previous one… except it didn’t. Turns out TP-Link has two separate Smart Home brands, and while they share the same account management, they use different apps and have different integration support. The Christmas tree uses a Kasa-branded plug, while the one I ordered was a Tapo-branded plug — the latter does not work with Home Assistant, which was a bit annoying, while the former worked well, when I used it.

When TP-Link then also got in the news for having “fixed” a vulnerability that allowed Home Assistant to control the Kasa plugs on the local network, I got annoyed enough that I decided to do something about it. (Although it’s good to note that nowadays Home Assistant works fine with the “fixed” Kasa plugs, too.)

I asked once again help to Srdjan who has more patience than me to look up random IoT projects out there, and he suggested me the way forward… which turned out a bit more tortuous than expected, so let me try to recount it here, showing shortcuts as much as I can at the time of writing

The Hardware

Things start a bit iffy to begin with — I say that because the hardware I ended up getting myself was Gosund SP111 from Amazon, after checking reviews that they were still shipping an older version of the hardware that can be converted with tuya-convert (I’ll get back to the software in a moment). There’s plenty of warnings about the new models, but in a very similar fashion to the problem with CGG1 sensors, there’s no way to know beforehand what version of a firmware you’re going to find.

But in particular, SP111 are not Gosund engineering’s own work. Tuya is a company that build “white label” goods for many different brands, and that’s how these plugs were made. You can find them under a number of different brand names, but they all share the same firmware and, critically, the same pairing and update keys.

One of the best part about these Tuya/Gosund plugs is that they are tiny in the sense of taking very little space. They are less than half the size of my first Kasa-branded TP-Link smart plug, and even when compared with the newer Tapo-branded one they are quite slimmer. This makes for adding them easily to places that are a bit on the constrained side.

The Firmware

I did not buy these devices to run with the original firmware. Unlike the CGG1 and pretty much everything else I’m running at home, this time I went straight for the “I want to run ESPHome on them.” The reason for that was on one side, I’m annoyed and burnt by the two TP-Link devices, and from the other, the price of Hue-compatible smart plugs was annoyingly high. That would have been my default alternative.

Of the various important things to me, Home Assistant support grown from a “meh, sure” to “yeah I want that” — and in part because I did manage to set up scripts for quite a bit of tools at home through it. One of my original concerns (wanting to still be able to control most of these features by voice) is taken care of by signing up for Nabu Casa, which provides integration for both Google Assistant and Alexa, so even with the plug running a custom and maybe janky, firmware, getting it to work with the rest of the house would be a breeze.

There’s other options for open source firmware for the SP111, as well as other smart plugs. Tasmota is one of the most commonly used for this, and there’s another called ESPurna. I have not spent time investigating them, because I already have one ESPHome device running, and it was easier to lower my cognitive load to use something I already kinda know. After all, the plan is to replace the Kasa plug once we take out the Christmas tree, which would remove not one but two integrations from Google and Amazon (and one from Home Assistant).

All of these options can be flashed the first time around with tuya-convert, even though the procedure ended up not being totally clear to me — although in part that was caused by my… somewhat difficult set up. This was actually part of the requirements. Most smart home devices appear to be built around ESP8622 or ESP32, which means you can, with enough convincing, flash them with ESPHome (I’m still convincing my acrylic lamp circuit board), but quite a few require you have physical access to the serial port. I wanted something I could flash without having to take it apart.

The way this works, in very rough terms, is that the Tuya-based devices can be tricked into connecting to a local open network, and from there with the right protocol, they can be convinced to dump their firmware and to update it with an arbitrary firmware binary. The tuya-convert repository includes pretty much all the things you need to set this up, neatly packaged in a nearly user-friendly way. I say nearly, because as it turns out there’s so much that can go wrong with it, that the frustration is real.

The Process.

Part 1: WiFi.

First of all, you need a machine that has a WiFi adapter that supports Access Point mode (AP mode/hostapd mode, those are the keywords to look for it). This is very annoying to know for sure, because manufacturers tend to use the same model number across hardware revisions (that may entirely change the chipset) and countries — after all, Matthew ended up turning to Xbox One controller adapters! (And as it turns out, he says they should actually support that mode, with a limited range.)

The usual “go-to” for this is to use a laptop which also has an Ethernet port. Unfortunately I don’t have one, and in particular, I don’t have a Linux laptop anymore. I tried using a couple of Live Distros to set this up, but for one reason or another they were a complete bust (I couldn’t even type on Ubuntu’s Terminal, don’t even ask.)

In my case, I have a NUC on top of my desk, and that’s my usual Linux machine (the one I’m typing on right now!) so I could have used that… except I did disable the WiFi in the firmware when I got it, since I wired it with a cable, and I didn’t feel the need to keep it enabled. This appears to have been a blessing in disguise for a while, or I would have been frustrated at one of the openSUSE updates in December, when a kernel bug caused the system to become unusable with the WiFi on. Which is what happened when I turned the WiFi on in the firmware.

Since I don’t like waiting, and I thought it would generally be a good idea to have at least one spare USB WiFi dongle at home (it would have turned useful at least once before), I went and ordered one on Amazon that people suggested might work. Except it probably got a hardware revision in the middle, and the one I received wasn’t suitable — or, some of the reports say that it depends on the firmware loaded on it; I really didn’t care to debug that too much once I got to that stage.

Fast forward a few weeks, the kernel bug is fixed, so I tried again. The tuya-convert script uses Docker to set up everything, so it sounded like just installing Docker and docker-compose on my openSUSE installation would be enough, right? Well, no. Somehow the interaction of Docker and KVM virtual machines had side effects on the networking, and when I tried I both lost connectivity to Home Assistant (at least over IPv6), and the tuya-convert script kept terminating by itself without providing any useful information.

So instead, I decided to make my life easier and more difficult at the same time.

Part 1.1: WiFi In A Virtual Machine

I didn’t want Docker to make a mess of my networking setup. I also wasn’t quite sure of what tuya-convert would be doing on my machine (yes, it’s open source, but hey I don’t have time to audit all of it). So instead of trying to keep running this within my normal openSUSE install, I decided to run this in a virtual machine.

Of course I need WiFi in the VM, and as I said earlier, I couldn’t just pass through the USB dongle, because it wouldn’t work with hostapd. But modern computers support PCI pass-through, when IOMMU is enabled. My NUC’s WiFi supports hostapd, and it’s sitting unused, since I connect to the network over a cable.

The annoying part was that for performance issues, IOMMU is disabled by default, at least for Intel CPUs, so you have to add intel_iommu=on for the option of passing through PCI devices to KVM virtual machines to be available. This thread has more information, but you probably don’t need all of it, as that focuses on passing through graphic cards, which is a much more complicated topic.

The next problem was what operating system to run in the VM itself. At first I tried using the LiveDVD of openSUSE — but that didn’t work out: the Docker setup that tuya-convert uses is pretty much installing a good chunk of Ubuntu, and it runs out of memory quickly, and when it does it throws a lot of I/O errors from the btrfs loaded into memory. Oops.

Missing a VM image of openSUSE Tumbleweed, I decided to try their image for JeOS instead, which is a stripped down version meant to be used in virtualized environments. This seemed to fit like a glove at first, but then my simplicity plans got foiled by not realizing that usually virtualized environments don’t care for WiFi. Although the utter lack of NetworkManager or any other WiFi tooling turned out to be handy to make sure that nothing tried to steal the WiFi away from tuya-convert.

In addition to changing the kernel package with a version that cares about WiFi drivers, you need to install the right firmware package for the card. After that, at least the first part is nearly taken care of — you will most likely need a few more tools, such as Git and your editor of choice. And of course, Docker and docker-compose.

And then, do yourself a favour, and turn off firewalld entirely in your virtual machine. Maybe I should have said earlier “Don’t let the VM be published to the Internet at large”, but I hope if you get to try to pass-through a WiFi device, you knew better than doing that anyway. The firewall is something that is not obvious is going to get in your way later when you set to run tuya-convert, and it’ll make the whole setup fail silently in the hardest way possible to debug.

Indeed, when I looked for my issues I found tons of posts, issues, blogs all complaining about the same symptom I had, which was all caused by having a firewall in place. The tuya-convert script does a lot of things to set up stuff, but it can’t easily take down a firewall, and that is a biggie.

Indeed, and I’ll repeat that later, at some point the instructions tell you to connect some other device to their network and suggest otherwise it might not be working. My impression is that this is done because if it doesn’t work, you shouldn’t try taking the next steps yet. But the problem is that there is no note anywhere to help you if it doesn’t work — and the reason for it failing is likely the firewall stopping the DHCP server from receiving the requests. Oops.

Part 2: The Firmware Blob

ESPHome configurations are… sometimes very personal. I have found one for the SP111 on the Home Assistant forums and adapted it, but… I don’t really feel like recommending that one. So I’m afraid I won’t take responsibility for how you configure your ESPHome firmware for the plug.

Also, once you have ESPHome on the device, changing the config is nearly trivial, from the Home Assistant integration, so I feel it’s important to have something working at first, and then worry about perfecting it.

I think someone will be confused here on why am I jumping on configuring the firmware blob before we got to convert the device to use it. The reason for that is that you want to have the binary file (either built locally or generated with the Home Assistant integration and downloaded), and you put it into tuya-convert/files/, you will be able to directly flash that version, without going through the intermediate step of using one of the bundled firmware just to be able to update to an arbitrary firmware. But to do that, it needs to happen before you complete the Docker setup.

So, find yourself a working config for the device on the forums (and if you find one that is maintained and templated, so that one can just drop it in and just configure the parameters, please let me know), and generate your ESPHome firmware from there.

Also note that the firmware itself identifies the specific device. This means you cannot flash more than one device with the same firmware or you’ll have quite the headache to sort them out afterwards. Not saying it isn’t possible, but I just found it easier to make the firmware for the devices I was going to flash, and then load each one. As usual, my favourite tool to remember what is what would be my label maker, so that I don’t mix up which one I flashed with which binary.

Part 3: The Conversion

Okay so here’s the inventory of what you should have by this point before we move on to the actual conversion:

a virtual machine with a passed-through WiFi card that is supported by hostapd;
an operating system in the VM with the drivers for the WiFi, Docker, docker-compose, and no firewall;
a checkout of tuya-convert repository;
one or more ESPHome firmware binary files (one per device you want to flash), in the files/ directory of the checkout.

Only at this point you can go and follow the instruction of tuya-convert: create Docker image, setup docker-compose, and run the image. The firmware files need to be added before creating the docker image, because docker-compose does not bind the external files/ directory at all.

Once the software starts, it’ll ask you to connect another device that is not the plug to the WiFi. I’m not entirely sure if it’s just for diagnostics, but in either case, you should be able to connect to the network — the password should be flashmeifyoucan, although I don’t think I’ve seen that documented anywhere except when googling around for other people having had issues with their process.

If you try this from your phone, you should be prompted to login into the WiFi network through a captive portal — the portal is just a single page telling you that the setup is completed. If your phone gets stuck in the “Obtaining IP Address” phase, like mine did, make sure you really took down the firewall. This got me stuck for a while because I thought that the Docker itself controlled the whole firewall settings — but that does not appear to be the case.

Final Thoughts

I guess that this guide is not going to be very useful, with the new versions of the SP111 not being supported by tuya-convert (and not clear if it can be supported), but since I have two plugs still unused, it helps me to have written down the process to avoid getting myself stuck again.

The plugs appear to have configurable sensors for voltage, amperage, and total wattage used — and the configuration of those is why I’m not comfortable sharing the config I’m using: I took someone’s from a forum post but I don’t quite agree with some of the choice made, some of the values appear fairly pointless to me.

Voltage monitoring would have been an useful piece of information when I was back in Italy — those who read the blog a long time ago might remember that the power company over there didn’t really have any decent power available. Over here it feels like it’s very stable, so I doubt we’ll notice anything useful with these.

Having Smart Home devices that don’t rely on cloud services is much more comfortable than otherwise. I do like the idea of being able to just ask one of the voice assistants to turn off the subwoofer while I’m playing Fallout 76, for sure — but it’s one thing to have the convenience, and another to depend on it to control it. And as I said some time ago, I disagree with the assertion that there cannot be a secure and safe IoT Smart Home (and yes, “secure” and “safe” are two separate concepts).

As for smart plugs in particular? I’m still not entirely sold, but I can see that there definitely are devices where trying to bring the smart in the device is unlikely to help. Not as many though — it’s still a problem to find something that cannot be served better by more fine-grained control. In the case of the subwoofer, most of the controls (volume, cross-over, phase) are manual knobx on the back of the device. Would it have made sense to have a “smart subwoofer” that can tweak all of those values from the Home Assistant interface? I would argue yes — but at the same time, I can see in this case an expense of £10 for a smart plug beats the idea of replacing the subwoofer entirely.

I honestly have doubts about the Christmas tree lights as well. Not that I expect to be able to control them with an app, but the “controller” for them seems to be fairly standard, so I do expect if I search AliExpress for some “smart” controller for those I will probably find something — the question is whether I would find something I can use locally without depending on an external cloud service from an unknown Chinese brand. So maybe I’ll go back to one of my oldest attempts at electronics (13 years ago!) and see what I can find.

By the way, if you’re curious what else I am currently planning to use these smart plugs on… I’m playing with the idea of changing my Birch Books to use 12V LEDs – originally meant for Gunpla and similar models – and I was thinking that instead of leaving it always-on, I can just connect it with the rest of the routines that we use to turn the “living” items on and off.

Kind Software

This post sprouts in part from a comment in my previous disclaim of support to FSFE, but it’s a standalone post, which is not related to my feelings towards FSFE (which I already covered elsewhere). It should also not be a surprise to long time followers, since I’m going to cover arguments that I have already covered, for better or worse, in the past.

I have not been very active as a Free Software developer in the past few years, for reasons I already spoke about, but that does not mean I stopped believing in the cause or turned away from it. At the same time, I have never been a fundamentalist, and so when people ask me about “Freedom 0”, I’m torn, as I don’t think I quite agree on what Freedom 0 consists of.

On the Free Software Foundation website, Freedom 0 is defined as

The freedom to run the program as you wish, for any purpose (freedom 0).

At the same time, a whole lot of fundamentalists seem to me to try their best to not allow the users to run the programs as they wish. We wouldn’t, otherwise, be having purity tests and crusade against closed-source components that users may want to actually use, and we wouldn’t have absurdist solutions for firmware, that involve showing binary blobs under the carpet, and just not letting the user ever update them.

The way in which I disagree with both formulation and interpretation of this statement, is that I think that software should, first of all, be usable for its intended purpose, and that software that isn’t… isn’t really worth discussing about.

In the case of Free Software, I think that, before any licensing and usage concern, we should be concerned about providing value to the users. As I said, not a novel idea for me. This means that software that that is built with the sole idea of showing Free Software supremacy, is not useful software for me to focus on. Operating systems, smart home solutions, hardware, … all of these fields need users to have long-term support, and those users will not be developers, or even contributors!

So with this in mind, I want to take a page out of the literal Susan Calman book, and talk about Kind Software, as an extension of Free Software. Kind Software is software that is meant for the user to use and to keep the user as its first priority. I know that a number of people would make this to be a perfect overlap and contrast, considering all Free Software as Kind Software, and all proprietary software as not Kind Software… but the truth is that it is significantly more nuanced than that.

Even keeping aside the amount of Free Software that is “dual-use” and that can be used by attackers just as much as defenders – and that might sometimes have a bit too much of a bias towards the attackers – you don’t need to look much further than the old joke about how “Unix is user friendly, it’s just very selective of who its friends are”. Kind software wouldn’t be selective — the user use-cases are paramount, any software that would be saying “You don’t do that with {software}, because it is against my philosophy” would by my definition not be Kind Software.

Although, obviously, this brings us back to the paradox of tolerance, which is why I don’t think I’d be able to lead a Kind Software movement, and why I don’t think that the solution to any of this has to do with licenses, or codes of ethics. After all, different people have different ideas of what is ethical and what isn’t, and sometimes you need to make a choice by yourself, without fighting an uphill battle so that everyone who doesn’t agree with you is labelled an enemy. (Though, if you think that nazis are okay people, you’re definitely not a friend of mine.)

What this tells me that I can define my own rules for what I consider “Kind Software”, but I doubt I can define them for the general case. And in my case, I have a mixture of Free Software and proprietary software in the list, because I would always select the tools that first get their job done, and second are flexible enough for people to adapt. Free Software makes the latter much easier, but too often is the case that the former is not the case, and the value of a software that can be easily modified, but doesn’t do what I need is… none.

There is more than that of course. I have ranted before about the ethical concerns with selling routers, and I’ve actually been vocal as a supporter for law requiring businesses to have their network equipment set up by a professional — although with a matching relaxation of the requirements to be considered a professional. So while I am a strong believer in the importance of OpenWRT I do think that trying to suggest it as a solution for general final users is unkind, at least for the moment.

On the other side of the room, Home Assistant to me looks like a great project, and a kind one to it. The way they handled the recent security issues (in January — pretty much just happened as I’m writing this) is definitely part of it: warned users wherever they could, and made sure to introduce safeties to make sure that further bugs in components that they don’t even support wouldn’t introduce this very same problem again. And most importantly, they are not there to tell you how to use your gadgets, they are there to integrate with whatever is possible to.

This is, by the way, the main part of the reason why I don’t like self-hosting solutions, and why I would categorically consider software needing to be self-hosted as unkind: it puts the burden of it not being abused on the users themselves, and unless their job is literally to look after hosted services, it’s unlikely that they will be doing a good job — and that’s without discussing the fact that they’d likely be using time that they meant to be spending on something else just to keep the system running.

And speaking of proprietary, yet kind, software — I have already spoken about Abbott’s LibreLink and the fact that my diabetes team at the hospital is able to observe my glucose levels remotely, in pretty much real-time. This is obviously a proprietary solution, and not a bug-free one at that, and I’m also upset they locked it in, but it is also a kind one: the various tools that don’t seem to care about the expiration dates, that think that they can provide a good answer without knowing the full extent of the algorithm involved, and that insist it’s okay to not wait for the science… well, they don’t sound kind to me: they not just allow access to personal data, which would be okay, but they present data that might not be right for people to take clinical decisions and… yeah that’s just scary to me.

Again, that’s a personal view on this. I know that some people are happy to try open-source medical device designs on themselves, or be part of multi-year studies for those. But I don’t think it’s kind to expect others to do the same.

Unfortunately, I don’t really have a good call to action here, except to tell Free Software developers to remember to be kind as well. And to think of the implications of the software they write. Sometimes, just because we’re able to throw something out there, doesn’t mean it’s the kind thing to do so.

Video: unpaper with Meson — pytesting our way out

Another couple of hours spent looking at porting Unpaper to Meson, this time working on porting the tests from the horrible mix of Automake, shell, and C to a more manageable Python testing framework.

I’ll write up a more complex debrief of this series of porting videos, as there’s plenty to unpack out of all of them, and some might be good feedback for the Meson folks to see if there’s any chance to make a few things easier — or at least make it easy to find the right solution.

Also, you can see a new avatar in the corner to make the videos easier to recognize 😀 — the art is from the awesome Tamtamdi, commissioned by my wife as a birthday present last year. It’s the best present ever, and it seemed a perfect fit for the streams.

And as I said at the end, the opening getting ready photo stars Pesto from Our Super Adventure, and you probably saw it already when I posted about Sarah and Stef’s awesome comics.

As a reminder, I have been trying to stream a couple of hours of Free Software every so often on my Twitch channel — and then archiving these on YouTube. If you’re interested in being notified about these happening, I’m usually announcing them with a few hours to spare (rarely more than that due to logistics) on Twitter or Facebook.

Progress Logging and Results Logging

There is one thing that my role as software mechanic seems to get me attracted to, and that’s the importance of logging information. Logging is one of those areas that tend to bring up opinions, and with the idea of making this into a wider area of observability, it brought on entire businesses (shout out to friends Honeycomb.io). But even in smaller realities, I found myself caring about logging, setting up complex routing with metalog, or hoping to get a way to access Apache logs in a structured format.

Obviously, when talking about logging in bubbles, there’s a lot more to consider than just which software you send the logs to — even smaller companies nowadays need to be careful with PII, since GDPR makes most data toxic to handle. I can definitely tell you that some of the analysis I used to do for User-Agent filtering would not pass muster for a company at the time of GDPR — in a very similar fashion as the pizzeria CRM.

But leaving aside the whole complicated legal landscape, there’s a distinction in logs that I have not seen well understood by engineers – no matter where they are coming from – and that is the difference between what I call progress logging and results logging. I say that I call them this way, because I found a number of other different categorizations of logs, but none that matches my thoughts on the matter, and I needed to give it names.

Distinctions that I did hear people talk about are more like “debug logs” versus “request logs”, or “text logs” versus “binary logs”. But this all feels like it’s mixing media and message, in too many cases — as I said in my post about Apache request logs, I would love for structured (even binary would do) request logs, which are currently “simple” text logs.

Indeed, Apache (and any other server) request logs to me fit neatly in the category of results logging. They describe what happened when an action completed: the result of the HTTP request includes some information of the request, and some information of the response. It provides a result of what happened.

If you were to oversimplify this, you could log each full request and each full response, and call that results logging: a certain request resulted in a certain response. But I would expect that there is a lot more information available on the server, which does not otherwise make it to the response, for many different reasons (e.g. it might be information that the requestor is not meant to find out, or simply doesn’t have to know, and the response is meant to be as small as possible). In the case of an HTTP request to a server that act as a reverse proxy, the requestor should not be told about which backend the request was handled by — but it would be an useful thing to log as part of the result.

When looking at the practicality of implementing results logging, servers tend to accrue the information needed for generating the result logs in data structures that are kept around throughout the request (or whatever other process) lifetime, and then extract all of the required information from them at the time of generating the log.

This does mean that if the server terminates (either because it’s killed, the power goes off, or the request caused it to crash), and the result is not provided, then you don’t get a log about the request happening at all — this is the dirty secret of Apache request logs (and many other servers): they’re called request logs but they actually log responses. There are ways around this, by writing parts of the results logs as they are identified – this helps both in terms of persistence and in terms of memory usage (if you’re storing something in memory just because you should be logging it later) but that ends up getting much closer to the concept of tracing.

Progress logs, instead, are closer to what is often called shotgun debugging or printf() debugging. They are log statement emitted as the code goes through them, and they are usually free-form for the developer writing the code. This is what you get with libraries such as Python’s logging, and can assume more or less structured form depending on a number of factors. For instance, you can have a single formatted string with maybe the source file and line, or you may have a full backtrace of where the log event happened and what the local variables in each of the function calls were. What usually make you choose between the two is cost, and signal-to-noise ratio, of course.

For example, Apache’s mod_rewrite has a comprehensive progress log that provides a lot of details of how each rewrite is executed, but if you turn that on, it’ll fill up your server’s filesystem fairly quickly, and it will also make the webserver performance go down the drain. You do want this log, if you’re debugging an issue, but you most definitely don’t want it for every request. The same works for results logs — take for instance ModSecurity: when I used to maintain my ruleset, I wouldn’t audit-log every request, but I had a special rule that, if a certain header was provided in the request, would turn on audit-logging. This allowed me to identify problems when I was debugging a new possible rule.

Unfortunately, my experience straddling open-source development and industry bubbles means I don’t have overall good hopes for an easy way to implement logging “correctly”. Both because correctly is subjective, and because I really haven’t found a good way to do this that scales all the way from a simple tool like my pdfrename to a complex Cloud-based solution. Indeed , while the former would generally be caring less about structured logs and request tracing, a Cloud software like my planned-and-never-implemented Tanuga would get a significant benefit from using OpenTelemetry to connect feed fetching and rendering.

Flexible and configurable logging libraries, such as are available for Python, Ruby, Erlang, and many more, provide a good “starting point” but by experience they don’t scale well between in and out of an organization or unit. It’s a combination of problems similar to the schema issue and the RPC issue: within an organization you can build a convention of what you expect logs to be, and you can pay the cost of updating the configuration for all sorts of tools to do the right thing, but if you’re an end user, that’s unlikely — besides, sometimes that’s untested.

So it makes sense that, up to this day, we still have a significant reliance on “simple”, unstructured text logs. They are the one universally accessible way to provide information to users. But I would argue that we should be better off to build an ecosystem of pluggable, configurable backends, where the same tool, without being recompiled or edited, can be made to output simple text on the standard error stream, or to a more structured event log. Unfortunately, judging by how the FLOSS world took the idea of standardizing services’ behaviours with systemd, I doubt that’s going to happen any time soon in the wide world… but you can probably get away with it in big enough organizations that control what they run.

Also, for a bit of fun related tidbit, verbose (maybe excessively so) progress logging is what made my reverse engineering the OneTouch Verio so easy: on Windows the standard error is not usually readable… unless you run the application through a debugger. So once I did that, I could see every single part of the code as it processed the requests and responses for the device. Now, you could think that just hiding the logs by default, without documenting the flag to turn them on would be enough — but as it turns out, as long as the logging calls are built into a binary, it’s not too hard to understand them while reverse engineering.

What this is meant to say is that, just because easy access to logs is a great feature for open source tools, and for most internal tools in companies and other institutions, the same cannot be said for proprietary software: indeed, the ability to obfuscate logs, or even generate “encrypted” logs, is something that proprietary software (and hardware) thrive on: it makes it harder to reverse engineer. So it’s no surprise if logs are a complicated landscape with requirements that are not only divergent, but at times opposite, between different stakeholders.

Video: unpaper with Meson — From DocBook to ReStructured Text

I’m breaking the post-on-Tuesday routine to share the YouTube-uploaded copy of the stream I had yesterday on Twitch. It’s the second part of the Unpaper conversion to Meson, which is basically me spending two hours faffing around Meson and Sphinx to update how the manual page for Unpaper is generated.

I’m considering trying to keep up with having a bit of streaming every weekend just to make sure I get myself some time to work on Free Software. If you think this is interesting do let me know, as it definitely helps with motivations, to know I’m not just spending time that would otherwise be spent playing Fallout 76.

unpaper: re-designing an interface

This is going to be part of a series of post that will appear over the next few months with plans, but likely no progress, to move unpaper forward. I have picked up unpaper many years ago, and I’ve ran with it for a while, but beside a general “code health” pass over it, and back and forth on how to parse images, I’ve not managed to move the project forward significantly at all. And in the spirit of what I wrote earlier, I would like to see someone else pick up the project. It’s the reason why I create an organization on GitHub to keep the repository in.

For those who are not aware, unpaper is a project I did not start — it was originally written by Jens Gulden, who I understand worked on its algorithms as part of university. It’s a tool that processes scanned images of text pages, to make them easier to OCR them, and it’s often used as a “processing step” for document processing tools, including my own.

While the tool works… acceptably well… it does have a number of issues that always made me feel fairly uneasy. For instance, the command line flags are far from standard, and can’t be implemented with a parser library, relying instead on a lot of custom parsing, and including a very long and complicated man page.

There’s also been a few requests of moving the implementation to a shared library that could be used directly, but I don’t feel like it’s worth the hassle, because the current implementation is not really thread-safe, and that would be a significant rework to make it so.

So I have been having a bit of a thought out about it. The first problem is that re-designing the command line interface would mean breaking all of the programmatic users, so it’s not an easy decision to take Then there’s been something else that I learnt about that made me realize I think I know how to solve this, although it’s not going to be easy.

If you’ve been working exclusively on Linux and Unix-like systems, and still shy away from learning about what Microsoft is doing (which, to me, is a mistake), you might have missed PowerShell and its structured objects. To over-simplify, PowerShell piping doesn’t just pipe text from one command to another, but structured objects that are kept structured in and out.

While PowerShell is available for Linux nowadays, I do not think that tying unpaper to it is a sensible option, so I’m not even suggesting that. But I also found out that the ip command (from iproute2) has recently received a -J option, which, instead of printing the usual complex mix of parsable and barely human readable output, generates a JSON document with the same information. This makes it much easier to extract the information you need, particularly with a tool like jq available, that allows “massaging” the data on the command line easily. I have actually used this “trick” at work recently. It’s a very similar idea to RPC, but with a discrete binary.

So with this knowledge in my head, I have a fairly clear idea of what I would like to have as an interface for a future unpaper.

First of all, it should be two separate command line tools — they may both be written in C, or the first one might be written in Python or any other language. The job of this language-flexible tool is to be the new unpaper command line executable. It should accept exactly the same command line interface of the current binary, but implement none of the algorithm or transformation logic.

The other tool should be written in C, because it should just contain all the current processing code. But instead of having to do complex parsing of the command line interface, it should instead read on the standard input a JSON document providing all of the parameters for the “job”.

Similarly, there’s some change needed to the output of the programs. Some of the information, particularly debugging, that is currently printed on the stderr stream should stay exactly where it is, but all of the standard output, well, I think it makes significantly more sense to have another JSON document from the processing tool, and convert that to human-readable form in the interface.

Now, with a proper documentation of the JSON schema, it means that the software using unpaper as a processing step can just build their job document, and skip the “human” interface. It would even make it much easier to write extensions in Python, Ruby, and any other language, as it would allow exposing a job configuration generator following the language’s own style.

Someone might wonder why I’m talking about JSON in particular — there’s dozens of different structured data formats that could be used, including protocol buffers. As I said a number of months ago, the important part in a data format is its schema, so the actual format wouldn’t be much of a choice. But on the other hand, JSON is a very flexible format that has good implementations in most languages, including C (which is important, since the unpaper algorithms are implemented in C, and – short of moving to Rust – I don’t want to change language).

But there’s something even more important than the language support, which I already noted above: jq. This is an astounding work of engineering, making it so much easier to handle JSON documents, particularly inline between programs. And that is the killer reason to use JSON for me. Because that gives even more flexibility to an interface that, for the longest time, felt too rigid to me.

So if you’re interested to contribute to an open source project, with no particular timeline pressure, and you’re comfortable with writing C — please reach out, whether it is to ask questions for clarification, or with a pull request to implement this idea altogether.

And don’t forget, there’s still the Meson conversion project which I also discussed previously. For that one, some of the tasks are not even C projects! It needs someone to take the time to rewrite the man page in Sphinx, and also someone to rewrite the testing setup to be driven by Python, rather than the current mess of Automake and custom shell scripts.

NewsBlur Review

One of the very, very common refrain I hear in my circles, probably because my circles are full of ex-users of it, and at the same time of Googlers and Xooglers, is that the Internet changed when Google Reader was shut down, and that we would never be able to come back. This is something that I don’t quite buy out right — Google Reader, like most of the similar tools, was used only by a subset of the general population, while other tools, such as social networks, started being widely used right around the same time.

But in the amount of moaning about Google Reader not existing anymore, I rarely hear enough willingness to look for alternatives. Sure there was a huge noise about options back then, which I previously called the “Google Reader Exodus“, but I rarely hear of much else. I see tweets going by of people wishing that Reader still existed, but I don’t think I have seen many willing to go out of their way to do something about it.

Important aside here: while I did work at Google when Reader was shut down in effect, the plan was announced in-between me signing my contract and my start date. And obviously it was not something that was decided there and then, but rather a long-term decision taken who knows how long before. So while I was at Google for the “funeral”, I had no saying, or knowledge, of any of it.

Well, the good news is that NewsBlur, which I have started using right before the Reader shut down, is still my favourite tool for this, it’s open source, and it has a hosted service that costs a reasonable $36/year. And it doesn’t even have a referral program, so if you had any doubt of me shilling, you can vacate it now.

So first of all, NewsBlur has enough options for layout that look so much like Google Reader “of back then” — before Google+ and before losing the “Shared Stories” feature. Indeed, it supports both its own list of followers/following, and global sharing of stories on the platform. And you don’t even need to be an user to follow what I share on it, since it also automatically creates a blurblog, which you can subscribe to with whatever you want.

I have in the past used IFTTT to integrate further features, including saving stories to Pocket, and sharing stories on Twitter, Facebook, and LinkedIn. Unfortunately while NewsBlur has great integration, IFTTT is now a $4/month service, which does not have nearly enough features for me to consider subscribing to, sorry. So for now I’m talking about direct features only.

In addition to the sharing features, NewsBlur has what is for me one of the killer features: the “Intelligence Trainer”. Which is not any type of machine learning system, but rather a way for you to tell NewsBlur to hide, or highlight, certain content. This is very similar to a feature I would have wanted twelve years ago: filtering. Indeed, this allowed me to hide my own posts from Gentoo Universe – back when I was involved in the project – and to only read Matthew’s blog posts in one of the many Planets he’s syndicated, like I wanted. But there’s much more to it.

I have used this up to this day to hide repetitive posts (e.g. status updates for certain projects being aggregated together with blogs), to stop reading authors that didn’t interest me, or wrote in languages I couldn’t read. But I also used the “highlighting” feature to know when a friend posted on another Planet, or to get information about new releases or tours from metal bands I followed, through some of the dedicated websites’ feeds.

But where this becomes extremely interesting is when you combine it with another feature that nowadays I couldn’t go without, particularly as so much content that used to be available as blogs, sites, and feeds is becoming newsletters: it’s the ability to receive email newsletters and turn them into a feed. I do this for quite a few of them: the Adafruit Python for Microcontrollers newsletter (which admittedly is also available through their blog), the new tickets alerts from a bunch of different venues (admittedly not very useful this year), Tor.com, and Patreon.

And since the intelligence trainer does not need to have tags or authors to go along, but can match a substring in the title (subject), this makes it an awesome tool to filter out certain particular messages from a newsletter. For instance, while I do support a number of creators on Patreon, a few of them share all their public videos as updates — I don’t need to see those in the Patreon feed, as I get them directly at source, so I can hide those particular series from the Patreon feed for myself. And instead, while I can wait for most of the Tor.com releases, I do want to know quickly if they are giving away a free book, or if there’s a new release from John Scalzi that I missed. And again, the highlighting helps me there: it makes a green counter appear next to the “feed”, that tells me there’s something I want to look at sooner, rather than later.

As I said the intelligence trainer doesn’t have to use tags — but it can use them if they are there at all. So for instance for this very blog, if I were to post something in Italian and you wouldn’t be able to read it, you could train NewsBlur to hide posts in Italian. Or if you think my opinions are useless, you can just hide those, too.

But this is not where it ends. Beside having an awesome implementation of HTTP, which supports all bandwidth-saving optimizations I know of, NewsBlur thinks about the user a lot more than Google Reader would have. Whenever you decide to do some spring cleaning of your subscription, NewsBlur will send you by email an OPML file with all of your subscribed feed before you made the first change (for the day, I think). That way you never risk deleting a subscription without having a way to find it agian. And it supports muting sites, so you don’t need to unsubscribe not to get a high count of unread posts of, say, a frequent flyers’ blog during a pandemic.

Plus it’s extremely tweakable and customizable — you can choose to see the stories as they appear in the feed, or load into a frame the original website linked by the story, or try to extract the story content from the linked site (the “reader mode”).

Overall, I can only suggest to those who keep complaining about Google Reader’s demies, that it’s always a good time to join NewsBlur instead.