Reverse Engineering an LG Aircon Control Panel — Fit It All Together

Since we moved to this flat, one of my wishlist item was to have a way to control the HVAC (heat, ventilation, and air conditioning) without having to get out of bed, or go to a different room. A few months ago, I started on a relatively long journey of reverse engineering the protocol that the panel used so that I could build my own controller, and I’ve been documenting pretty much every step along the way, either on Twitter, this blog, or Twitch. As I type this, while I can’t say that the project is all done and dusted, I’ve at least managed to get to the point where I’m reaping the benefits of this journey.

You may remember that at the end of the previous chapter in this saga I was looking at controlling the actual HVAC with a Python command line tool. I built that on stream, to begin with, and while I didn’t manage to make it work the way I was hoping (with a curses-based UI), I did manage to get something that worked to emulate both the panel and (to a minor point) the HVAC itself.

I actually used this “in production” (that is, to control the air conditioning in my home office) for a little over a week. The way I was using it was through a Beagle Bone Black which I had laying around for a long while – I regret not signing up for the RISC-V preview, but honestly I wouldn’t have had time – connected through an USB-to-UART adapter (a CH340 based one, because they can actually do 104bps!), and a breadboarded TLIN1206 on a SOP-8-to-DIP adapter. A haphazard and janky setup if you’d ever seen one, but it was controllable by phone… with JuiceSSH and Tailscale.

While absolutely not ergonomic, this setup still allowed me to gain the experience needed with the protocol, in order to send the PCB design to manufacture. I did that about a week in, and as usual, JLCPCB‘s turnaround is phenomenally fast, and by the following Monday I had my boards.

The design is pretty much the same as the one I spoke about in part two, with two bus transceiver blocks, a 3.3V DC-DC converter, the quad-NAND to make sure the code cannot send data to the bus if the physical switch is turned to “disable”.

While the design proven it needs a little bit more work to be optimal, it’s a great starting point because it just works fine. And while I did originally plan to have this support the original panel with the switch turning on “pass-through mode”, I decided that for the moment, this is not needed, nor desired.

My original intention was to allow the physical switch to just prevent the custom controller from talking to the HVAC engine, and let the original panel take over. Unfortunately this does not work: the panel is not entirely stateful, but there is a bit in the command packet that says “Listen to me, I’m changing configuration” — and if the configuration in the panel and the HVAC don’t match when that bit is not set, an error state is introduced.

This basically means I wouldn’t be able to seamlessly switch between the old panel and my custom controller. Instead what I’m going to be doing if I need to is to first flip the switch to disable the custom logic, and then connect the panel to the secondary bus. That way it would initialize directly on the bus. If I do need to re-design this board, I’m going to make the switch more useful, and add a power disconnection so that it can be all connected without power on at all.

There was another reason why I originally planned to support the original control panel: it reports a temperature. I thought that I would be able to use that temperature as a “default” sensor, while still allowing me to change the source of the temperature at the time of configuration. But the panel’s temperature sensor is pretty much terrible, and it’s only able to measure 16°C~30°C, plus it’s easily fooled by… a lamp. So not exactly something you can call reliable for this use, and not something I would care to add to my monitoring.

To my own astonishment, my first attempt at soldering the full board was successful — two out of three of the boards work like a charm, the third one is a bit iffy, but it might be a not perfect component into it. I’ll have to see, but also I don’t want to make a new respin now that even some of the components I need are getting harder (and more expensive) to find. Take the JST ZR connection: in the picture above you an see it white rather than cream — that’s because it’s not an official JST part but an AliExpress clone that fits the same footprint, and I could actually buy.

Custom ESPHome Climate

Once I had the board, I rigged up a quick test bed on my desk with a breadboard and another CH340 adapter (I have a few around, they are fairly cheap and fairly versatile), and started off to complete the Custom Climate component.

Since I started this project, the ESPHome Climate API actually changed a couple of times, particularly as Nabu Casa is now sponsoring the project, and its development moved to a monthly release kind of deal. Somehow the Climate components were among those that most required work.

But the end result is that the API was clearer, and actually easier to implement, by the time I had this ready. So I wrote down the code to generate the six bytes packets I needed to send, and ran it against the emulator… and it seemed to work fine. I had wired this with a test component in Home Assistant and I could easily change the mode, the temperature, and everything else just fine, and I could see in my emulator that it was getting the right data in just fine.

At that point I was electrified, and I thought it would just be a matter of putting it to the wall and see it work. You can imagine my disappointed when I called in my wife to assist me in my victory… and it didn’t work. And I was ready to detach all of it to spend the next day debugging what was going on, until I realized that I forgot to send the “settings changed” flag. I have to say that the protocol turns out to be a bit more complex than I expected it to be, and I should probably write a bit more documentation about it, not just scatter it around the (now multiple) implementations.

After that, I actually went ahead and replaced all three of the control panels with my custom ones, and connected them to Home Assistant. That turned out to be a much easier prospect than anything else we have been trying up to now: we can decide which temperature reading to use to control the room, rather than being stuck to the silly temperature sensor in the panel, and we can use the two-point set temperature in the way most modern smart thermostat can (which the panel didn’t support, despite having an “auto mode” that would turn on cooling or heating “as needed”).

The first couple of days lead to a bit of adjustments being necessary — including implementing a feature that my wife requested: when not cooling or heating, the original panel would enter “fan only” mode. Which I enjoy for myself in the office, but bothers my wife. The original panel does not have an option to turn the fan off — but I could implement that in the custom controller. This allows us to keep our thermostat on the heat-cool mode most of the time, and just make sure the range is what we actually care for.

I also made the mistake at first of not counting on hysteresis. That turned out to be a bit more annoying to implement, not in the matter of code, but in the matter of the logic behind it — but it should now be working: it means that there is more friction to change the state of the air conditioning, which means the temperature is not as constant, but it should be significantly easier to run. To be honest I was impressed by how stable the temperature was when I left it to short cycle…

Home Assistant Integration

This was probably the simplest part of the whole work! Nabu Casa is doing an awesome job at keeping the two projects integrated very well, and with the help of “packages” configuration, replicating the configuration for the three separate boards took basically no time.

The only problem I had was that I couldn’t seem to be able to flash the first ESPHome firmware onto my ESP32 devkits using the WebSerial support. I have used it multiple times in the past, particularly to update my BLE Bridge, which I just need to connect to my main workstation rather than the standalone power supply, to upgrade, but for the pristine ESP32 devkits it didn’t work out quite as well.

The UI is very similar to the one that Google Home exposes for Nest thermostats where they do support air conditioning. And indeed, with the addition of the Home Assistant Cloud service, the same UI shows up for these thermostats.

And at that point it was just a matter of configuring the expected range of operation, both for the “daytime” and for the “night scene”. Which is one of the reasons why I wanted to have thermostat that we can control with Home Assistant.

You see, some time ago I set up so that we have a routine phrase (“To the bedroom”) and Flic buttons in both the living room and the bedroom, that prepare us to get to bed: turn off the TV, subwoofer, air fresheners, lights everywhere except bedroom and bathroom, set the volume of the bedroom speaker for the relaxing night sound, and so on.

Recently, thanks to a new Dyson integration, it also been setting the humidifier to raise the humidity in the bedroom (it gets way too dry!), and turn on the night mode on the other purifiers, which has been a great way not just to make it easier not to forget things, but also to save us from leaving the humidifier running 24/7: it’s easier for us to keep it running overnight at a higher humidity, than trying to keep it up during the whole day.

Now, with the climate controls in place, we can also change the temperatures before going to bed, rather than turning it off, which is the only option we had before. And this is a big deal, because particularly for the living room, we don’t want it to get too scorching hot even if we’re not there: it’s where the food cupboards, among other things, and during the heatwave we exceeded the 30°C for a couple of hours every other day. Being able to select different ranges while we’re still sleeping gives us a bit more safety, without having to keep running the air conditioning overnight.

Features And Remote Control

Similarly, our scheduled morning routine, configured to go off together with the alarm clock, can scale back the range in my home office to something suitable for my work day (it can get warm fast with computers and stuff running, and the door closed for meetings), so I don’t have to over-run the office in the morning when having my first meeting: it starts automatically, and it welcomes me with a normal, working temperature for my first meeting.

The final point is that, since we can actually set a very wide range on the thermostats, and rely on much more accurate thermometers that are not restricted to the 16°C to 30°C range, we can leave the thermostat on, with a very wide range, when we’re not actually going to be at home. This is particularly interesting now that we might be able to travel again, at least to see families and friends — we don’t want to leave the air conditioning or heating running all the time, but we also want to have some safeguards against the temperature dropping or raising out of control. This became very clear when the CGG1 in our winter garden hit the 50°C ceiling it could report while we were out playing Pokémon Go and we couldn’t turn on the air conditioning at all — thankfully the worst result of that was that the body of one of the fake candles we have in the winter garden for ambiance melted… and now it looks even more realistic as a candle (it also damaged the moving flame part, but who cares.)

Now, the insulation of this flat is not great. In particular the home office tends to drop down close to freezing temperatures in the winter, because the window does not seal even when closed. This means I can’t easily follow Alec’s suggestion on energy storage. But having a bit more control on automation does make it easier to keep the temperature in the flat more stable. In the winter, I expect we’ll make sure that we keep a minimum temperature overnight to avoid having to force a much higher differential when we wake up, for instance.

There’s a few features that I have not yet implemented, and that I definitely should look into implementing soon. To start with, as soon as summer gives away to autumn, we’re likely going to want to be using the dehumidifier more — without turning on the heat pump, particularly in the living room as we cook and eat. Since the CGG1 provides humidity readings together with temperature, it means I can set up an automation that, if nothing else is running, turns the dehumidifier on if the humidity reaches a certain set point, for instance.

There’s also two switches that I have not implemented yet, but should probably do, soon. One turns on resistive heating – and this will make sense again if you watch Alec’s video on heat pumps – while the other has to do with the Plasma Filter.

What’s a plasma filter? That’s a good question, and one that I’m not sure I have the right answer for. I know that this is something that the original control panel suggests is present in our HVAC, although I have no way to know for sure (we don’t have the manual of the actual engines). The manual for the PQRCUDS0 says that «you can use the plasma function» but also states «If the product is not compatible with the Plasma function,
it will not do the Plasma function even though the indicator is turned on.» This suggests that unlike other features like the swirl/swing, it’s not part of the feature query that the panel sends at turn on.

When googling further to look for information about LG’s plasma filter, I did find another manual, for an actual unit, rather than the control panel. Not the unit we’ve got, I think, but at least an unit. And this one has a description for the plasma functionality:

Plasma filter is a technology developed by LG to get rid of microscopic contaminants in the intake air by generating a plasma of high charge electrons. This plasma kills and destroys the contaminants completely to provide clean and hygienic air.

This is quite a bit interesting — and next to this, a video from LG refers to this as a “Plasma/Ionizer”, which pretty much suggested me that this is one of Big Clive’s favourite toys: ozone generators. Which makes sense given that one of his favourites is a Sharp Plasmacluster.

Code And Next Steps

First of all, the code and the board designs are all available on my GitHub. I originally considered making this a normal component for ESPHome, but since it relies on a very custom board, it doesn’t feel like the right thing to do. It does mean I need to manually keep in mind all the various changes in APIs, but hopefully that will not take too much of my time.

As I said previously, I have not actually implemented the panel bus handler — the panel will enter into error mode if it does not get an expected reply from the HVAC engine, so connecting it right now would not work at all, except if you were to disable the actual ESP32 control. I’m likely going to leverage that behaviour to test some more error handling in the future.

I would like to put a box around the board — right now it’s literally stuck to the wall with some adhesive feet, in all three rooms. And while the fixed red LED is not too annoying overnight, it is noticeable if you wake up in the middle of the night. My original idea was to find someone who can help me 3D print a box that fits on the same posts the panel fits, and provide a similar set of posts for the original panel. But it also involved me finding a way to flip the switch without taking it down the wall.

But since figuring out 3D printing with no experience is going to take a lot of investment, I am not going to take a look at that option until I’m absolutely certain I’m not changing anything in the design. And I know I would want to change the design a little bit.

First of all, I want to have a physical cut-off for the connection — since the power to run the ESP32 module comes from the same cable controlling the HVAC, the only way to turn off the power is to disconnect the cable, right now. Having a physical switch that just disconnects the power and data would just make it easier to, say, replace the devkit module.

Similar, as I said, the panel is not that useful to keep running all the time. So instead I would change the switch implementation to keep the panel off most of the time, and only power it on when disabling the ESP32. This would also save some components, since there would be no need to have the second bus transceiver and its passives.

I’m also wondering if it would make sense to have some physical feedback and access, in addition to the Home Assistant integration and the voice assistant controls: in particular I’m considering having an RGB LED on the board to tell the current action being taken (optionally, I wouldn’t want to have that in the bedroom, as it would be way too bright) together with a button to at least turn the HVAC “soft off”.

Finally, there’s a couple of optimizations that could be done to make the board a bit cheaper. One of the capacitor is ceramic, but could be replaced with a polymer one for ⅓ of the price, and the TVS diodes pair (which were actually a legacy part of the design, recommended by the MCP2021, but not in the reference designs for the TLIN1027) could be replaced with a single integrated TVS diode — it would just be “a bit” harder to solder by hand in TO-236 packages.

These are all minor though — the main cost behind the board is actually the ESP32-DEVKIT-32D that it’s designed around. It would be much cheaper to use only the ESP32 WROOM module, and not have the USB support components on the board. But I have had bad experiences with trying to integrate that in my designs, so I’m feeling a bit sceptical of going down that route — it also would mean that a botched board sacrifices the whole module (I did sacrifice two or three of those already) unless you get very good at desoldering them (or you have a desoldering station).

So most likely it will take me a few months before I actually get to the point of trying building a 3D printed cover for it. With a bit of luck by then we’ll be back in an office at least part of the time, and I can get someone to teach me how to use the 3D printer there.

Also as a final note — the final BOM for the boards suggest that building one of them costs around £25 or so, without the case. As I said there’s a few cost saving measures that I could take for the next round — though it’s questionable, because it would require me to get more components I don’t currently have. Of course the actual cost of building three of them turned out to be… significantly higher.

I think this is the part that sometimes it’s hard to explain to people who have not had this type of experience: the BOM costs are only one of the problems you need to solve — you can really screw up a project by choosing the wrong components and bringing the BOM price too high… but a low BOM cost does not make for a cheap project to finish, particularly when you’re developing it from nowhere.

In this case, if I wanted to tally up the cost of building these custom thermostat panels, I would have to, at the very least, count the multiple orders from DigiKey from which I ordered the wrong components (like the two failed attempt at using Microchip’s LIN bus transceivers rather than the TLIN, or the discrete UARTs with their own set of passives). But then there’s the cost of having all of the various tools that were needed to get this all done. Thankfully most of those tools have been used (and sometimes abused) for different projects, but they are a good metaphor for the cost of R&D that many products need — so it makes sense that what you end up buying costs more than the “simple” expense of the BOM. So keep that in mind next time you see an open source hardware device costing more than what you expect it to.

Reverse Engineering an LG Aircon Control Panel — Low-Speed Serial Issues

In the previous part of the LG aircon reverse engineering, I gave the impression that the circuitry design was a problem mostly solved. After all, I found working LINbus transceivers, and I figured out a way to keep one of them “quiet” when not in use. And indeed, when I started drafting the post, it was supposed to be followed by a description of the protocol I identified, and should have been talking about how I wrote tools to simulate this protocol to test the implementation without actually touching the HVAC unit.

And that’s what I set out to do myself on a live stream, nearly two months ago now — the video says “Attempt 2” because on the first, short stream I ended up showing on stream my home wifi password and, even though it’s not likely that a bad actor would show up at my place to take over the wifi, it’s not good opsec, so I stopped the stream, rotated the password of all the devices at home, and came back to the stream.

So what happened? Well, while I was trying to figure out how to build an ESPHome custom component for the “climate” platform, but trying to send sequence of bytes through the serial port appeared to not work correctly: instead of being sent at the selected polling frequency, they would be “bunched up” together, sending three bytes instead of one, or twelve instead of six. It worked “fine” if I flushed the serial port, but the flush operation would then take longer than the time between the commands I wanted to send, so that didn’t sound like a good plan.

As you can imagine from the title, this particular problem only happened with the slow, 104 8n1 configuration that the LG aircon needs — it didn’t happen at all with higher baudrates such as 9600, which suggested the problem was related to the timing of the connection, which is not uncommon: a lot of UART implementations that include FIFOs tend to define some timing based on the timing of a “space” or of a full character.

What also suggested me that, is that someone, somewhere, was complaining that the ESP32 couldn’t do the slow speed that this aircon needs, and that they preferred using the ESP8266 because that one came with a software serial implementation. Unfortunately, I cannot find anymore where I read that, to link it there and to point out that the code for the ESP8266 software serial actually works without significant modifications on the ESP32 — it’s just that the lack of need for it means it’s not readily available.

So indeed, I managed to get the ESP8266 software serial to work… except for the fact that it was not quite reliable. At 104 bps (which is the speed the aircon protocol needs) sending a six bytes sequence (which is the size of an aircon packet) takes about half a second — add another half second for the response (which is also six bytes), and you have a recipe for a disaster: one second every two seconds (which is the frequency of command exchange between panel and HVAC) would be spent just on serial communication — anything else happening during those time and messing up the timing meant bad communication.

Another nearly-software-based alternative I attempted, and also kind-of worked, was using the RMT peripheral. This is the remote control peripheral included in ESP32 — and the reason why Circuit Python made it harder to send pulse trains on FeatherS2: it’s no longer just implemented in software, but it relies on hardware buffers to allow sending and receiving pulse trains. It’s awesome to offload that, but it also comes with limitations. In particular, while I did manage to implement a 104 bps serial transmission through this interface, it would only allow me one serial pair rather than two, severely limiting what I could be doing with the aircon board.

Content Warning: though I’m personally aiming at following more modern conventions, the terminology in use by datasheet and other content that I’m about to link to still use older, less inclusive terminology.

UART — But Discretely

So instead, I used my overcomplication skill to come up with an alternative: discrete UARTs! You see, back in the days when personal computers came with serial ports, and before chipsets started having everything into a single physical chip, and even before most of the functionality of basic peripherals was merged into a Super I/O chip, we had multiple competing discrete UART chips available. The most common one being the 16550, at least for my personal experience. These are still available, and you can indeed still use the 16550, although to do so you need to use a lot of I/O lines, as 16550 and compatible have usually a parallel I/O interface, which is suitable for ISA bus connection, but not so suitable for a microcontroller even when it’s quite generous with its GPIO lines.

As an alternative, I started looking into using a SC16IS741A, which is a I²C (and SPI) chip. Instead of using a lot of separate I/O lines for sending commands and data to it, you send it with the usual two wire interface, and it internally decodes it into whatever format it needs. You may wonder what the difference is, between using the actual UART and sending it over to an I²C hardware UART — the answer is a bit complicated to explain theoretically, but I think a visualization of it will go a long way to explain:

What you see here is a screenshot from the Saleae Logic software capturing three I/O lines: the I²C bus (SCL above, SDA below), and the TX line of the discrete UART. This is a very much not optimized transaction that shows the sending of one byte at the 104 bps configuration that my aircon control needs. It’s one order of magnitude slower to send the byte than to set up the UART to send the message and send it, and this is with a relatively slow bus as I²C is. And it doesn’t scale linearly, even.

Basically, the discrete UART allows offloading the whole process of keeping up with the timing, and it does so in a very efficient way. Receiving is even more interesting, because it does not require the microcontroller to pay attention to the received bytes until it’s ready to process them, and maintain them in the FIFO in the meantime. But this kind of features already exist in most microcontrollers (often referred to “hardware UART”), and when they work, that’s awesome… but clearly sometimes they don’t quite work.

This particular device would be useful on boards based around the older ESP8266 micro, as that only has a single hardware UART, and is used for the logging. With one (or more) of this or similar chips you would be able to control a much wider number of serial-controlled devices, and that makes them valuable.

Unfortunately, ESPHome does not really have a way to provide an uart bus outside of the core platform, at least right now. If I did end up working more down this particular route, I would probably have paid more attention to integrate it — it’s not hard to provide the same interface so that it would be a drop-in replacement, but it does require some reshuffling of the core hierarchy so I don’t think I can pull that out just yet.

Writing a Device Driver, a Component, a Library

In either case, whether you want to integrate this directly in ESPHome, or use it from another software stack, you need to implement its own command set. This is what you usually refer to as a “device driver” for operating systems such as Linux and Windows, as it provides access to the underlying device with some standard interface. And indeed, the Linux kernel has a driver for a set of related of NXP UART peripherals: sc16is7xx.

Now, while the NXP-provided datasheet has all the information needed to program for this chip, having the source reference on Linux made things significantly easier, particularly because unless you know what to look for, you most likely will misread the datasheet. I have some (though not much) experience with I²C devices, but there were a few things that ended up confusing me enough that I wasted hours down the wrong route.

The first problem was figuring out the addressing convention. I²C addresses should, by convention be 7 bit. While the protocol sends a whole byte for addressing, it uses the last bit in it to specify whether you’re issuing a read or a write. But despite this being the common convention, and the one that ESPHome, CircuitPython, and just about anything else expects you to use, some datasheet do not follow that, and provide the full 8 bit “address”. You can see more details on that on the Total Phase website, which was instrumental for me to get to the bottom of why things kept disagreeing with what I was writing.

Once the peripheral was addressed, the question to answer was about the registers addresses. In my first attempt at configuring the chip I was falling short. A quick look through the Linux sources told me that I was missing a left shift of the register address… which made me go “Huh?” for a while. Indeed, the datasheet provides explicit tables explaining the register addressing: registers have a 4-bit address, but are set in the 3:6 bits of a byte, with bit 7 (MSB) being used in SPI only to select between reading and writing to the register. Of the remaining three bits, two are used to select between channels — because some of the matching chips by NXP include more than one UART on board, though unfortunately I couldn’t find one that I could easily work with on a breadboard that did. The last one (LSB)… is not used. It’s always off and reserved. But more interestingly, the Linux driver only shifts the address by two, not by three like I had to. So I’m wondering if this does mean that this chip is only mostly compatible with the ones I was looking.

So after one full day of figuring out how to properly run my component over ESPHome, I decided I needed something for prototyping this I2C faster.

Enter Circuit Python

By now you probably know that I do like Circuit Python. And as it turns out, I already have written some code for I²C in Circuit Python when I extend the MCP230xx library to include the older ’16 model. So it didn’t feel too odd to go ahead and use a Trinket M0 with Circuit Python to play around with the UART.

The choice of the Trinket M0 over the more capable Feathers was not random: while the Trinket has a physical UART and the pins to use it, it’s also a very tiny device. The fact that you can use multiple physical UARTs through the I²C bus allows a significant expansion of the I/O abilities of that class of microcontrollers.

At the end, I not only ended up writing a CircuitPython compatible library that allowed me to use the UART, but also re-writing it to leverage the Adafruit_CircuitPython_Register library, making it significantly easier to add support for more features.

The library supports an interface that is nearly identical to the one provided by the built-in serial, although I don’t think theré sa way to make sure it really is, because similarly to ESPHome it doesn’t look like Circuit Python ever consider the need to support UARTs that are not part of the original hardware design, understandably so, as these discrete UARTs are still fairly uncommon.

But I went one step further: when I read the datasheet the first time I wasn’t sure just how strong the suggestion for 1.8432 MHz crystals was, for the divisor. Turns out it’s not strong at all, so the whole amount of crystals I bought at that frequency are not particularly helpful. Worse yet, it turns out I don’t need any clock, because even the Trinket M0 is able to create a 50% duty cycle PWM output at a frequency that is high enough to use as driving clock for the UART.

That means that I can fit, in a half sized breadboard, the whole circuitry I need, including only two passives (the pull-up resistors on the SCL/SDA lines), providing the clock as a PWM output from the Trinket while also piggybacking its reset line. This was a surprisingly good setup, and would actually allow me to control the two sides of my aircon (panel and hvac) if I was still going with the discrete UART idea.

But it turns out I really don’t need any of that: ESP32’s UARTs worked out just fine at the end — at least in the most recent firmware as uploaded by ESPHome, so I decided to set aside again the UARTs and try instead to control the aircon at least with an USB-to-UART adapter. But that’s a story for another post.

Bonus Chapter: Dual-UART chips

As I said earlier, there are some options out there for multiple UART chips that would be interesting to use for cases like mine, in which I need two independent, yet identically configured UARTs. Dual UART chips are not uncommon, but I²C controlled ones are.

If you look around for I²C Dual-UART options, you most ilkely will end up on the DFRobot DFR0627 which is an “IIC to Dual UART Module” — IIC being the name you’ll find used on Chinese products to refer to I²C (it’s like TF card instead of SD card, don’t ask me.)

So why did I not even consider this particular option? Well, the first issue is that this is a full module, that uses the Gravity connector (which is similar to, but as far as I know not compatible with, the Stemma QT connector that Adafruit uses), the second issue is that there’s no documentation to go with how to use it.

Since I want to, at the end of the whole process, have a printed board I can just hang on the wall (possibly with a 3D Printed case, but that’s further along the way), I need to be able to get the components I want in retail-enough options that I can buy them and solder them in myself. I also need to be able to control those components with arbitrary software.

The DFRobot modules tend to have Arduino components, which you may still be able to use for ESPHome, but you wouldn’t be able to use with Circuit python during the more iterative side of the project. Since these components are open source you could go ahead and reverse engineer it from those, but it would be much easier to develop for something that has a datasheet and some documentation.

Indeed, the DFRobot website does not even tell you what chip is on the module. though if you look around in the forums you can find a reference to WEIKAI WK2132-ISSG, which is available through LCSC and comes with a datasheet. In Chinese.

If you just look at the pictures, you can at least confirm that this device is similar in functionality to what the NXP part I’ve been working with provides, except for the fact that it does not have the full RS-232 style CTS/RTS lines. So it really would be an interesting part if I ever decided to go back to the idea of using discrete UARTs, but it would require at the very least for me to get one of my Chinese-reading friends to translate enough of this 28 pages datasheet to be able to tell what to do. That is unlikely.

Extending the Saleae Logic

One of the reasons why I have always appreciated FLOSS is the ability to adapt, modify, and reuse tools and code and designs to build just what you need. Over the years I have done that many, many times, including writing ruby-elf to run my analysis and taking over unpaper to handle the scanned document.

What I have not, until lately, realized, is the usefulness of building your own physical tools, or rather I have figured that out by watching Adam Savage videos on YouTube, but I didn’t actually build physical tools until pretty much last year, when I started the Birch Books project.

Now, even though Saleae Logic Pro is not open source, neither in its hardware nor its software, I was recommended it many years ago, and I think it’s one of my best purchases in a long time, because it’s very compact, the software works great on Linux and Windows (which is handy because my Windows machine always had a lot more RAM), and the support folks are awesome at addressing issues (I reported a few wishlists, and a couple of bugs, and I think the only one that took a long time was a licensing issue). And since supporting and maintaining those tools is not my main hobby, I’m okay with accepting some closed source tools if they allow me to build more open source.

(I have also ordered a Glasgow, obviously – I mean, Hector working on it made it kind of an obvious thing for me to do – but I have had the Logic Pro for multiple years and the Glasgow is expected to deliver next year, so…)

Tip-Ring-Ring-Sleeve Man-In-The-Middle

So anyway, last year I started looking at ways I could make my life easier, and since I was already getting stuff printed at JLCPCB (not a sponsor), I thought I would try to at least build one of the devices I needed: a board that would let me pass through two TRRS connectors while “copying” the signals to the Saleae. The end result is in the following pictures (two different versions of the same concept — originally I made the board a bit too long, and the connector didn’t fit quite right).

As you can see from the image on the right (or the bottom on mobile), the main use I have for this is to be able to observe the transmission between computers and glucometers, since that has been my main target for reverse engineering over the years, and very often they are connected via a TRS 2.5mm or 3.5mm serial plug, similar to how Osmocom has introduced the world of open hardware many years ago.

The board is simplistic, but also provides a few features to make my life easier. First of all, it uses TRRS connectors, because they are compatible with the good old TRS connectors, but also support more modern ones. Secondly, it’s only 3.5mm plugs on purpose: finding 2.5mm cables is annoying, but finding 3.5 to 2.5 and vice versa is easy, and that’s why I use it with those cables at the end.

Finally, killer feature for me is that switch, which selects between CTIA and OMTP ordering of the connectors. If you’re not aware of it, over time two separate standards have been in use for wiring four conductor TRRS cables, with one of them often incorrectly referred to as “Android” or “Samsung” and the other referred to as “Apple”. The CTIA/OMTP naming is the correct naming, and basically what this does is just changing which of the two pins of the connectors is provided as ground to the Saleae.

Oh yeah and eventually, I released it. I did that under 0BSD, because it’s a really obvious design and if someone wants to reuse it, I’m only happy. I have considered whether this should be released under the CERN Open Hardware License and I can’t imagine why, but if you want to make an argument for it I’m likely going to be swayed by it, feel free.

I also originally drawn plans for a USB-to-Serial adapter using CP2104 — in two versions, one that would just be a simpler USB-to-UART with no level shifting, and one that would be a full-blown level shifting RS-232 port. The latter was something that came to me as an idea after reading a lot Foone on Twitter, to the point that I sent them a some of the untested boards (because I got stuck with the move and stuff, so it took me months to get back to them!)

Unfortunately, something didn’t work out quite right. The serial adapter didn’t work out at all in my case, and the RS-232 one keeps browning out, probably because I skimped on the capacitors. I would be ready to get a respin to try this again, but the current chip shortage does not allow me to make more orders for SMT boards with that particular chip in it.

Serial Adapters Harnesses

Not quite related to the Saleae, but related to the recent work on my aircon reverse engineering, was starting to think about labelled wire harnesses for serial adapters. And thanks to Meowy reply I went down the rabbit hole looking at professional label makers.

Now, let me be clear, I have had a label maker for quite a long time. I have blogged about it in 2011, when even the simplest of the label makers (Dymo LetraTag) was to be considered a risky investment (that’s what happens when you’re self-employed, coming from a blue-collar family, and in the backwaters of everything — one day I need to write more about that). I still use that label maker, pretty much our whole spice cabinet depends on it.

But there’s a different class of label makers, namely electricians’ label makers, such as the one showed off in the Big Clive video on the side. Of all the features that he shows off on that video, though, it was missing showing off prints on heatshrink, and how you can use those to make those labelled harnesses as shown in the tweet.

So, while the original price I could see (£110) was a bit annoying to invest for something I wasn’t sure I would be using very often, following Clive’s advice of waiting for Toolstation having it at a discount (which it did, at £69) was worth it — well worth it for me. Though to be honest, I got some off-brand heatshrink cartridges.

So the first thing I did with these have been creating a wire harness for one of the USB serial adapters that I have been working on the aircon with:

For this particular adapter, I also used a 1×5 Dupont plastic block — I did not crimp the cables, they come from a box of male-to-female cables that I bought off AliExpress some time ago, though you can refer to Big Clive on crimping, too. One of the things he shows in this last video, is that you can just break the plastic tooth to release the metal Dupont connector. Which is what I did: I broke off the plastic teeth, and re-fit the already crimped cable into the 1×5 housing.

I could have just prepared my own cables, to be honest. But to be honest, if I wanted to do that I would probably grab some good quality soft cables like the one that comes with the Saleae Logic, but I don’t have any of that stuff at home.

The other important part with this is that I used male-to-female on the cables, because these are the cables I need right this moment, because I’m working on the air conditioning reverse engineering, and that means I need to connect the serial adapter straight into the breadboard.

So About That Breadboard

You may have noticed one thing I said: I’m currently working on the aircon reverse engineering with breadboards, so having the cables end with a male connector helps. If you’re familiar with the Saleae Logic, you would know that the harnesses it comes with are 2×4 dupont cables with a female end to connect to boards. Which works great when you’re connecting to I/O pins on a board, or using the provided probes — but not really if you’re doing work on a breadboard.

So in the same vein as I did for the serial adapter above, I also decided to go ahead and make my own set of additional harnesses. They are once again not perfect, as the cables are the same AliExpress cables I used for the other harness, but they will do the job for the time being.

I have kept the colours the same – or at least the closet that I could get to with the AliExpress cables – because the Logic software uses the colours to represent the channels. And that means I don’t need to worry anymore of matching colours with colours all over the place. On the other hand, since working with a breadboard means I don’t have that many different ground positions, I decided to only put one ground wire per block. This is also because I ran out of black cables in the pile of AliExpress ones (I should order more).

This turned out to be extremely useful. Being able to just grab the cables and plug them straight into the breadboard on one side, and the Logic Pro on the other made has been a huge timesaver during my work, both off-camera and during streams.

A Note On Board Design

After this experience, I also want to share some of the considerations I made my mind up on, after these trials and errors. The first is that it’s very hard to find more than 1×8 Dupont connectors. That means that in many, many cases it’s much easier to go ahead and use 2×5 connectors if you have multiple pins that need to be connected together.

It also makes sense to space them to “key” them as a single connector, rather than using multiple, out of line pin lines, which I have done in the past. Indeed, in the custom CP2102 that I spoke about earlier, I wouldn’t be able to have a single harness, and would rather need two. That was a bad idea for a design, and when I’m going to re-do it (because I am going to re-do it), I’ll make sure the pins are arranged 2×4 instead, so that it can be connected with a custom labelled harness… or one of the harnesses that comes with the Logic!

This is the kind of important and useful notes that I like reading, finding in videos, and I wish would be collated in more practical material for wannabe PCB designers. It definitely carries over on my current designs of the air conditioning control board.

Customizing The Software

In addition to the custom hardware dongles I’ve been playing with, I also started looking at using more advanced software for the analysis.

In the Software Defined Remote Control repository, I already had a binary that would be able to receive a CSV exported by Logic and interpret it. Unfortunately for this to be easily parsed from within Logic, I would need to write a C++ extension for it.

On the other hand, for the air conditioning, I just needed to write a Python High-Level Analyzer, which provides me with at least a bit of the understood meaning of the various bytes in the packets.

I hope that as time goes on, and I find myself reverse engineering different hardware, I will be able to build up a good library of various analyzers — hopefully sharing enough code between them. Which is something I definitely need to engage with Saleae on: it does seem like right now you cannot depend on external Python modules even from HLAs, but it would make sense to be able to use libraries like construct, or even higher level libraries for things like my aircon or some of the glucometers I have worked on before.

Reverse Engineering an LG Aircon Control Panel — Buses and Cars

This is part two of my tale of reverse engineering the air conditioning control panel in our apartment. See the first part for further details.

If you are binging on retrocomputing videos like I’ve been doing myself, you may have the wrong impression that a bus has to have multiple lines, like the ISA and PCI buses do. But the truth is that a single-wire bus is not unheard of or even uncommon. It just means that the communication needs to be defined in such a way that there’s no confusing as for who is sending at any given time. In this case, it’s clear that the control panel is sending six bytes which are immediately (and I do mean immediately) followed by six bytes response from the HVAC.

So the next step was to figure out what those six bytes where, and thanks to Saleae’s recent licensing of sample high-level analyzers, this became a piece of cake. While I’m not at liberty to share the code, at the time of writing, I ended up writing an analyzer that would frame together the 6 bytes from the panel and the 6 bytes from the HVAC. Once I had that, it was also easier to notice that the checksum byte was indeed the same as other LG protocols, it’s just that it applied separately to the two 6 bytes packets, which means there’s only five bytes in the message that need to be decoded.

A screenshot of the Logic 2 software showing an analyzed trace with the high level analyzer loaded.

With a bit of trial and error, I already decoded what I think will give me most of the important controls for my plan: how to change the mode between the aircon, heat pump, fan, dehumidifier, and how to change the fan speed. The funniest part is that the “Auto” mode is actually not a mode at all, and just means that the thermostat appears to be sending the “aircon” or “heat pump” as needed.

What got even more interesting, is that if you leave the control panel by itself, after a few minutes it appears to notice the lack of an HVAC connected, and goes into an error state where it alternates the display between “Ch” and “3”. Either it’s reporting its own channel for diagnostics (assuming it’s misconfigured) or it’s just showing a particular error status. In either case, that threw a spanner in my plans.

The first problem is that obviously you wouldn’t be able to connect the 12V data wire to the ESP32 directly. That’s kind of obvious: the ESP32 is a 3.3V microcontroller, and if you tried to use a 12V wire with it it’ll just… go. My original intent was to use two optocouplers: one to receive the data from the control panel, and the other to inject my messages onto the wire. But that won’t work quite the same way for a bus, and while I could try to build up the right circuitry with discrete components, I would have rather used a ready-made transceiver.

The problem with that the transceivers are made for specific buses, and so the first question is to find the right bus that is by LG. A lot of HVAC systems (particularly in industrial scale) use Modbus over RS-485 — I have experience with this since the second company I ever worked for is a multinational that works in the industrial HVAC sector, so I learnt quite a bit of how those fit together. But an RS-485 connection would require two wires, since it uses differential signaling, and that’s already excluded.

Going pretty much by Google searches, I finally nailed down something useful. In the automotive industry, there’s a number of standards for on-board diagnostics (OBD). The possibly most famous (and nowadays most common) of those is the CAN bus, which is widely used outside that one industry, as well. LG is not using that. But one of the other protocols used is ISO 9141-2, which includes a K-Line bus on it, which according to Wikipedia is an asynchronous serial connection over a single bidirectional wire without handshakes — though it is using a 10.4kBd signal which is… exactly 100 times faster than the LG signal.

Through these, I found out about the LIN (Local Interconnect Network) bus, which is also used in automotive, specifies a higher level implementation on top of ISO 9141 compatible electrical signaling, but happens to be a good position to start the work with. Indeed, there are a number of LIN bus transceivers that are pretty oblivious of the addressing and framing on the protocol — on purpose, because the specifications have changed over the years. But what they are good for, is to connect to a 12V, high recessive bus, and provide microcontroller-leveled RX and TX signals.

An example of these transceivers is Microchip’s MCP2003, so I decided to set myself up to redesign the board based on that. But since the control panel also needed to receive “acknowledgements” from the HVAC, it meant that each “smart controller” needs two transceivers: one where it fakes the controller to the HVAC, and another one where it fakes the HVAC to the controller. And both of those needed to have the ability to just go into a “lurking” state where they wouldn’t be sending signals if I flipped a physical switch.

Screw It, I’m Doing It Live

So here’s where things got a bit more interesting in multiple directions. In the days just before this work, I was being asked a few pointers about reverse engineering — and unfortunately I don’t know how to “teach” RE, but I can at least “go through the motion”. After all, that was the more interesting part of my Cats Protection streaming week, so once the DigiKey order arrived with the transceivers and all the various passives to add around it, I decided to set up a camera, and try breadboarding the basic circuitry.

Now, setting aside the fact that I do not particularly enjoy streaming with an actual camera, and indeed the end results left a lot to be desired, the two hours stream was fairly productive. I found that the PL2303 USB-to-serial adapters actually work quite well at both 100 and 104 Bps, and that indeed the transceiver mostly works fine.

It also showed an interesting effect that I did not expect: as I said earlier, after a few minutes without getting an answer from the HVAC, the control panel enters into an error state (Ch/3). I assumed that what it needed was a valid packet from the HVAC, with checksum and information. Instead, it seems like just filling up a buffer, even with invalid packets, is enough to keep the control panel working: as I typed random words onto the serial port, while connected to the bus, the Ch/3 error vanished, and the panel went back to a working state.

This was surprising for one more reason: at least some of the packets sent from the HVAC to the panel had to include the capabilities the HVAC system has to begin with. The reason why I knew that is that the control panel appears to have a lot more functions when it’s running standalone, compared to when it’s installed on the wall. Things like a “power” fan mode for the aircon, the swiveling ventilation, and so on.

Spoiler: it turned out to indeed be the case: the first two commands sent from the panel to the HVAC appear to be some sort of inquiry, that provide some state to the panel to know which features are supported, including the heat-pump mode and the different fan speeds. But for now, let’s move on.

Before I could go and and try to figure out which bit related to which capability I hit a snag, which is what I got stuck at the end of the stream there: sending the character ‘H’ on the serial port (a very random character that just happens to be the start of the string “Hello, world!”) showed me something was… not quite right.

A screenshot from Logic2 showing a 0x68 sequence being interpreted as 0x6C.

This is not easy to see, beside for the actual value changing, but in the image above the first row (Channel 0) is the 12V bus (which you can read on the fourth line is actually 10V), the second and fifth rows (Channel 4) are a probe connected to the RXD pin of the MCP2003, and the third and sixth (Channel 5) are a probe connected to the TXD pin (which is in turn connected to the TXD of the USB-to-serial adapter).

Visibly, the problem is that somehow the bus went from “dominant” (0V) to “recessive” (12 10V) too fast, making the second and third bits look like 1s instead of 0s. But why? My first thought was that it was an electrical characteristic I missed – I did skimp on capacitors and diodes on my breadboarding – but after the stream terminated, I grabbed my Boox, and checked the datasheet more carefully and…

1.5.5.1 TXD Dominant Time-out

If TXD is driven low for longer than approximately 25 ms, the LBUS pin is switched to Recessive mode and the part enters TOFF mode. This is to prevent the LIN node from permanently driving the LIN bus dominant. The transmitter is reenabled on the TXD rising edge.

MCP2003/4/3A/4A Datasheet, DS20002230G, page 10

25ms is nearly exactly how long the dip to dominant state is on Channel 0 (and about the same on Channel 4): it’s also nearly exactly 2.5 baud.

A Note About Baudrate

I have complained loudly before of how I’m annoyed at people who think those younger than them know nothing and should just be made fun of. I don’t believe in that, and I think we should try our best to explain the more “antique” knowledge when we have a chance.

Folks who have been doing computers and modems well before me appear to love teasing people about the difference between “baudrate” and “bits per second”. The short version of that is that the baud rate relates to the speed of sending a single impulse, while the bits per second (bps) is (usually, but not always) meant to be taking the speed of the actual data transmitted. The relation between the two is usually fixed per protocol, and depends on how you send those bits.

In a asynchronous serial protocol (including RS-232 and this LG abomination), you define how you send your bits with an expression such as “8n1” or “7-odd-2” (also called the framing parameters) — or a number of other similar expression with different values in them. These indicate that each character sent is respectively eight or seven bits in size, that the parity is not present in the first case, and is odd in the latter, and that the first includes only one stop bit while the second is providing two. In addition to this, there’s always a single start bit.

8n1 is probably the most common of the framings, and that means you’re actually sending 10 bits for each character. A baudrate of 9600 Bd/s gives you a 960 bps raw connection, the 104 value for LG is the actual baudrate, as I can measure one of the impulses from the original control panel at 9.745ms — which actually would put it around 103 Bd/s.

Which is where my assertion that 25ms is nearly exactly 2.5 baud — 2.65 to be a bit more precise: you take the length (25) and divide it for the time needed to send a single baud (0.9745).

What this means in practicality is that the MCP2003 series (including the more modern MCP2003B that includes the same time-out behaviour) has a minimum baud rate as well as a maximum one. The maximum one is documented in the datasheet as 20 Kb/s, but the minimum is affected by this timeout: a frame of all zeros would be the worst case scenario in this condition, as the line would be asserted low (“dominant”) for the longest time. While theoretically you can define framings the way you prefer, the common configurations vary between 5 and 9 data bits per frame (though I would have no clue how to process the 9 bits per frame to be honest!) — which means that the maximum number of space (‘0’) baud would vary between 6 and 11.

Why six and eleven? Well, the “start” baud is also a space (logical zero) – which means that if your framing is 5n1, the 0x00 value would be sent with six “spaces”. And if you use nine data bits per frame with even parity, 0x000 would then be followed by a “space” in parity (to maintain the number of ‘1’ bits even), bringing it up to 11 (start, nine zeros, and parity).

The minimum baudrate for a certain framing configuration is thus calculated by dividing the maximum number of consecutive spaces the timeout in seconds (0.025), which leads to a minimum baudrate of 240 Bd/s for when using 5n1, 440 Bd/s for 9e1, and 360 Bd/s for the most commonly used 8n1 framing. Which is over three times faster than what these LG units are using.

I Need A New Bus Transceiver

Since I couldn’t use the MCP2003, I ordered a few MCP2021. Note that Microchip also says that these are not recommended in new designs, suggesting instead the ATA663232 — which as I’ll get to has all of the disadvantages of all the various options for LIN bus transceivers.

When I received the meter, I decided to take another stab at streaming setting up the emulator on camera:

If you watch the whole video you will see me at some point put a finger on the chip and yelp — turns out I ended up with a near-dead short on its embedded regulator. Thankfully, since the chip is designed for the automotive market, the stress did not cause it to fail at all, just… overheat. And as I showed on stream, I did manage to keep the control panel running with my “emulator”, although I did note some noise on the I/O towards the end.

So a little bit more exploration later told me that a) the PL2303 seems to be a bit unreliable with the 3.3V without tying the VREG with the 3.3V coming from the device, and b) even on the CH341 I would get some strange noise in addition to the signal. I think the reason for that is that the chip uses a comparator against its own regulator to decide whether the transmitter should be on. Since, as Monty and Hector suggested, it’s a bad idea to tie multiple regulators together, I decided that even the MCP2021 is not the transceiver I wanted.

Unfortunately, that made it harder to find the right transceiver. Microchip’s suggested replacement, the ATA6632xx series, has all of the disadvantages, as I said: it has the “TXD Dominant Timeout” feature (so it cannot send the 104bps signal I need to send), it includes a voltage regulator that cannot be disabled, and it is only available in VDFN package that is not possible to hand-solder.

On Digi-Key (which is by now my usual supplier), Microchip’s MCP20xx series are the only PDIP-8 through-hole components, so the next best thing is SOIC-8, which is surface mount (so not easily breadboardable) but still hand-solderable (with a steady hand, a magnifying glass, and a iron tip). Looking at those, I found at least two that fit.

ON Semiconductor’s NCV7327 was a very obvious choice because they explicitly say in the features list «Transmission Rate up to 20 kbps (No low limit due to absence of TxD Timeout function)», and it was the only one that I found explicitly note that the TxD Timeout imposes a floor to the speed (as I explained above). Unfortunately, the SOIC-8 version was not available at the time of order on Digi-Key, with a 22 weeks backorder.

So instead, I settled for Texas Instrument’s TLIN1027DRQ1. This is pretty much… the same. For what I can see, both ON’s and TI’s SOIC-8 devices are pin compatible, and they are nearly pin compatible with Microchip’s SOIC-8 variants, insofar as the power, bus, RXD, and TXD pins are in the same position.

There is, though, a rake just waiting for you there. The Enable/Chip Select pins on both the TLIN1027DRQ1 and the NCV7327 do not correspond to the MCP20xx Transmission Enable semantics, despite sharing the same position. With the MCP20xx you could leave a transceiver connected to a chatty bus, with the TXEnable off, and you would still receive the traffic from the bus.

But with the other two, you’re turning off the whole transceiver at once, which wouldn’t be too bad if it wasn’t that both of these pull TXD to ground (dominant), if you leave it unconnected. Again, this isn’t a big problem in by itself, as long as the firmware is told not to transmit when the bus is connected directly between the panel and the HVAC, nothing should be transmitted, right?

But this does break one assumption I was making: if I disable the smart controller board, I want to be able to remove the ESP32 devkit altogether. This is important because beside OTA (Over The Air) updates, I would need to be able to disconnect the ESP32 to update the firmware on it. Which means I don’t want to rely on the firmware being running and not holding the bus busy.

A schematic diagram of the Panel-side bus transceiver block.

So what I ended up adding to the design is a way for the bus selector to decide whether transmission is to be allowed on the transceiver. I think this is the first time I even consider the idea of using a 74-logic component in my designs (to the point that I had to figure out how to use that with the EAGLE-provided symbols — hint: use the invoke command), but this seemed to me as the easiest option to implement what I needed.

The tie-up-both-inputs for the NAND is literal textbook electronics, but turns out to work very well since the cheapest 74 logic NAND chip I found contains four of them, and I only need one other.

Note that of course this is only one of the “logical blocks” of the board — and actually not even the final form of it. As I get into more details later, you’ll find out that this only turned out to be one of the possible solutions, and (at the time of writing) there’s no guarantee that this is actually going to be the one I’m going to be using.

What’s Up With CircuitPython in 2021?

Last year, as I started going through some more electronics projects, both artsy and not, I have lauded the ease that Circuit Python adds to building firmware for sometimes not too simple devices, particularly when combined with Adafruit’s Feather line up of boards with a fixed interchangable interface.

As I noted last month, though, it does feel like the situation in the last year didn’t improve, but rather become more messy and harder to use. Let me dig a bit more into what I think is the problem.

When I worked on my software defined remote control, I stopped at a checkpoint that was usable, but a bit clunky. In this setup, I was using an Feather nRF52840, because the M0 alternatives didn’t work out for me, but I was also restarting it before each command was sent, because of… issues.

Indeed, I found my use case turned out to be fairly niche. On the M0, the problem was that the carrier wave signal used to send the signal to the Bravia TV was too off in terms of duty cycle, which I assume was caused by the underpowered CPU running the timers. On the nRF52840 the problem was instead that, if I started both transmitters (for the Bravia and for the HDMI switch), one of them would be stuck high.

So what I was setting up the stream to do (which ended up not working due to muEditor acting up), was to test running the same code on three more feathers: the Feather RP2040, the FeatherS2, and the Feather M4. These are quite more powerful than the others, and I was hoping to find that they would work much better for my use case — but turned out to tell me something entirely different.

The first problem is that the Circuit Python interfaces are, as always, changing. And they change depending on the device. This hit me before with the Trinket M0 not implementing time the same way as the Feather M0, but in this case things changed quite a bit. Which meant that I couldn’t use the same code I was using on the nRF Feather on the two newest Feathers: it worked fine on the M4, it needed some tweaking for the FeatherS2, and couldn’t work at all on the RP2040.

Let’s start with the RP2040, the same way I wanted to start the stream. Turns out the RP2040 does not have PulseOut support at all, and this is one of the problems with the current API selection in Circuit Python: what you see before you try and look into the details, is that the module (pulseio) is present, but if you were to break down the features supported inside, you would find that sending a train of pulses with that module is not supported on the RP2040 (at least as of Circuit Python 6.2.0).

Now, the good part is that RP2040 supports Programmable I/O (PIO), which would make things a lot more interesting, because the pulsing out could be implemented in the “pioasm” language, most likely. But that requires also figuring out a lot more than I originally planned on doing: I thought I would at least reuse the standard pulsing out for the SIRC transmitter, as that does need a 40 kHz carrier, and the harder problem was to have a non-carried signal for the switch (which I feed directly into the output of the IR decoder).

So instead I dropped off to the next in the list, the FeatherS2, which I was interested in because it also supports WiFi natively, instead of through the AirLift. And that was the even more annoying discovery: the PulseOut implementation for the ESP-S2 port doesn’t allow using a PWM carrier object, it needs the parameters instead. This is not documented — if you do use the PulseOut interface as documented, you’re presented with an exception stating Port does not accept PWM carrier. Pass a pin, frequency and duty cycle instead. At the time of writing that particular string only has two hits on Google, both coming from a Stanford’s course material that included the Circuit Python translation files, so hopefully this will soon be the next hit on the topic.

Unfortunately there is also no documented way to detect whether you need to pass the PWM or the set of raw parameters, which I solved in pysirc by checking for the platform the code is running in:

_PULSEOUT_NO_CARRIER_PLATFORMS = {"Espressif ESP32-S2"}
if sys.platform in _PULSEOUT_NO_CARRIER_PLATFORMS:
    pulseout = pulseio.PulseOut(
        pin=pin, frequency=carrier_frequency, duty_cycle=duty_cycle
    )
else:
    pwm = pulseio.PWMOut(
        pin, frequency=carrier_frequency, duty_cycle=duty_cycle
    )
    pulseout = pulseio.PulseOut(pwm)

Since this is not documented, there is also no guarantee that this isn’t going to be changing in the future, which is fairly annoying to me, but is not a big deal. But it also is not the only place where you end up having to rely on sys.platform for the FeatherS2. The other place is the pin names.

You see, one of the advantages of using Feathers for rapid prototyping, to me, was the ability to draw a PCB for a Feather footprint, and know that, as long as I checked that the pins were not being used by the combination of wings I needed (for instance the AirLift that adds WiFi support forces you to avoid a few pins, because it uses them to communicate), I would be able to take the same Circuit Python code and run it with any of the other feathers without worry.

But this is not the case for the FeatherS2: while obviously the ESP32 does not have the same set of GPIO as, say, the RP2040, I would have expected that the board pin definition would stay the same — not so. Indeed, while my code originally used board.D5 and board.D6 for the outputs, in the case of the FeatherS2 I had to use board.D20 and board.D21 to make use of the same PCB. And I had to do that by checking the sys.platform, because nothing else was telling.

The worst part of that? These pin definitions conflict between the FeatherS2 and the Feather M4. board.D20 is an on-board components, while board.D21 is SDA. On the FeatherS2, SDA is board.D10, and the whole right side of the pins is scrambled, which can cause significant conflicts and annoying debugging if using the same code in the same PCB with M4 versus S2.

This is what made me really worried about the direction of Circuit Python — when I looked at it last year it was a very good HAL (Hardware Abstraction Layer) that allowed you to write code that can work in multiple boards. And while some of the features obviously wouldn’t be available (such as good floating-point on the M0), those that would be were following the same API. Nowadays? Every port appears to have a slight different API, and different devices with the same form factor need different code. Which is kind of understandable, as the amount of supported boards increased significantly, but it still goes against the way I was hoping it would go.

Now, if the FeatherS2 worked for my needs, that would have been good enough. After all, I adapted pysirc to work with the modified API, so I thought I would be ready. But then something else caught my attention: as I said above, the signal sent to the switch doesn’t need to be modulated on top of a carrier wave. Indeed, if I did do that, the switch wouldn’t recognize the commands. So for that I would like a 100% duty cycle, but the FeatherS2 doesn’t have that as an option. Nor does it seem to be able to set a 99% cycle either. So I found myself looking at a very dirty signal compared to what I wanted.

Image
What was supposed to be a continuous 9ms pulse on Saleae’s Logic software.

Basically, the FeatherS2 is not fit for purpose when it comes to the problem I needed solving. So at the end of the day, I ended up just going back to what is in my opinion the best implementation for the form factor: the Feather M4. This worked fine for sending the two signals without needing to restart the Feather (unlike on the nRF), which shortened the time needed to send messages to TV and switch significantly (yay!) even though it still has a bit of a “dirty” pulse signal when running a 99% duty cycle at 10 Hz:

Image
A slightly noisy 9ms pulse followed by a train of noisy pulses.

But I’m still upset, because I feel Circuit Python is becoming less reliable on the “Write Once, Run Anywhere” suggestion that it was giving last year: you really need to make the code tailored for a particular implementation, instead of being able to rely on the already existing libraries.

Update 2021-05-26

Scott Shawcroft from Circuit Python got in touch as a follow up from this post, and it looks like some of the issues I reported here were either already on the radar (PulseOut for rp2040), re-opened following this post (PulseOut for ESP32-S2), or have now been addressed (Feather pin names). A huge thanks to Scott and the Circuit Python team all!

You can also see here that Scott is well aware of the importance of this all to fit together nicely — so hopefully the FeatherS2 is going to be a one-off special case!

Software Defined Remote Control

A number of months ago I spoke about trying to control a number of TV features in Python. While I did manage to get some of the adapter boards that I thought I would use printed, I hadn’t had the time to go and work on the software to control this before we started looking for a new place, which meant I shelved the project until we could get to the new place, and once we got there it was a matter of getting settled down, and then, … you got the idea.

As it turns out, I had one week free at the end of November — my employer decided to give three extra days on the (US) Thanksgiving week, and since my birthday was at the end of the week, I decided to take the remaining two days off myself to make it a nice nine days contiguous off. Perfect timeframe to go and hack on some projects such as that.

Also, one thing changed significantly since the time I started thinking about this: I started using Home Assistant. And while it started mostly as a way for me to keep an eye on the temperature of the winter garden, I found that with a bit of configuration, and a pull request, changing the input on my receiver with it was actually easier than using the remote control and trying to remember which input was mapped to what.

That gave me finally the idea of how to implement my TV input switch tree: expose it as one or more media players in Home Assistant!

Bad (Hardware) Choices

Unfortunately, as soon as I went to start implementing the switching code, I found out that I had made a big mistake in my assumptions: the Adafruit FT232H breakout board does not support PWM outputs, including the general time-based pulsing (without a carrier frequency). Indeed, while the Blinka library can technically support some of the features, it seems like none of the Linux-running platforms would be able to manage that. So there goes my option of just using a computer to drive the “fake remote” outputs directly. Well, at least without rewriting it in some other language and find a different way to send that kind of signals.

I looked around for a few more options, but all of it ended up being some compromise: MicroPython doesn’t have a very usable PulseOut library as far as I could tell; Arduino-based libraries don’t seem to allow two outputs to happen at roughly the same time; and as I’m sure I already noted in passing, CircuitPython lacks a good “secondary channel” to be instructed from a computer (the serial interface is shared with the REPL control, and the HID is gadget-to-host only).

After poking around a few options and very briefly considering writing my own C version on an ATmega, I decided to just go for the path of least resistance, and go back to CircuitPython, and try to work with the serial interface and its “standard input” to the software.

The problem with doing that is that the Ctrl-C command is intended to interrupt the command, and that means you cannot send the byte 0x03 un-escaped. At the end I thought about it, and decided that CircuitPython is powerful enough that just sending the commands in ASCII wouldn’t be an issue. So I decided to write a simplistic Flask app that would take a request over HTTP and send the command via the serial port. It worked, sort of. Sometimes while debugging I would end up locking the device (a Trinket M0) in the REPL, and that meant the commands wouldn’t be sent.

The solution I came up with was to reset the board every time I started the app, by sending Ctrl-C and Ctrl-D (0x03, 0x04) to force the board to reset. It worked much better.

Not-Quite-Remote Controlled HDMI Switch

After that worked, the problem was ensuring that the commands sent actually worked. The first component I needed to send the commands to was the HDMI switch. It’s a no-brand AliExpress-special HDMI switch. It has one very nice feature for what I need to do right now. It obviously has an infrared remote control – one of those thin, plasticky domes one – but it particularly has the receiver for it on a cord, which is connected with a pretty much standard 3.5mm “audio jack”.

This is not uncommon. Randomly searching Amazon or AliExpress for “HDMI switch remote” can find you a number of different, newer switches that use the same remote receiver, or something very similar to it. I’m not sure if the receivers are compatible between each other, but the whole idea is the same: by using a separate receiver, you can stick the HDMI switch behind a TV, for instance, and just make the receiver poke from below. And most receivers appear to be just a dome-encased TSOP17xx receiver, which is a 3-pin IC, which works great for a TRS.

When trying this out, I found that what I could do would be to use a Y-cable to allow both the original receiver and my board to send signals to the switch — at which point, I can send in my own pulses, without even bothering with the carrier frequency (refer to the previous post for details on this, it’s long). The way the signal is sent, the pulses need to ground the “signal” line (that is usually at 5V); to avoid messing up the different supplies, I paired it on an opto-coupler, since they are shockingly cheap when buying them in bulk.

But now that I tried setting this up with an input selection, I found myself not able to get the switch to see my signal. This turned out to require an annoying physical debugging session with the Saleae and my TRRS-to-Saleae adapter (that I have still not released, sorry folks!), which showed I was a bit off on the timing of the NEC protocol the switch used for the remote control. This is now fixed in the pysirc pysdrc library that generates the pulses.

Once I got the input selector working for the switch with the Flask app, I turned to Home Assistant and added a custom component that exposes the switch as a “media_player” platform. In a constant state of “Idle” (since it doesn’t have a concept of on or off), it allowed me and my wife to change the input while seeing the names of the devices, without hunting for the tiny remote, and without having to dance around to be seen by the receiver. It was already a huge improvement.

But it wasn’t quite enough where I wanted it to be. In particular, when our family calls on Messenger, we would like to be able to just turn on the TV selected to the right input. While this was partially possible (Google Assistant can turn on a TV with a Chromecast), and we could have tried wiring up the Nabu Casa integration to select the input of the HDMI switch, it would have not worked right if the last thing we used the TV for was the Nintendo Switch (not to be confused with the HDMI switch) or for Kodi — those are connected via a Yamaha receiver, on a different input of the TV set!

Enter Sony

But again, this was supposed to be working — the adapter board included a connection for an infrared LED, and that should have worked to send out the Sony SIRC commands. Well, except it didn’t, and that turned out to be another wild goose chase.

First, I was afraid that when I fixed the NEC timing I broke the SIRC ones — but no. To confirm this, and to make the rest of my integration easier, I took the Feather M4 to which I hard-soldered a Sony-compatible IR LED, and wrote what is the eponymous software defined remote control: a CircuitPython program that includes a few useful commands, and abstractions, to control a Sony device. For… reasons, I have added VCR as the only option beside TV; if you happen to have a Bluray player by Sony, and you want to figure out which device ID it uses, please feel free.

It might sound silly, but I remember seeing a research paper in UX from the ’90s of using gesture recognitions on a touchpad on a remote control to allow more compact remote controls. Well, if you wanted, you could easily make this CircuitPython example into a touchscreen remote control for any Sony device, as long as you can find all the right device IDs, and hard code a bunch of additional commands.

So, once I knew that at least on the software side I was perfectly capable of control the Sony TV, I had to go and do more hardware debugging, with the Saleae, but this time with the probes directly on the breadboard, as I had no TRS cable to connect to. And that was… a lot of work, to rewire stuff and try.

The first problem was that the carrier frequency was totally off. The SIRC protocol specifies a 40kHz carrier frequency, which is supposedly easier to generate than the 38kHz used by NEC and others, but somehow the Saleae was recording it as a very variable frequency that oscillated between 37kHz and 41kHZ. So I was afraid that trying to run two PWM outputs on the Trinket M0 was a bad idea, even if one of them was set to nought hertz — as I said, the HDMI switch didn’t need a carrier frequency.

I did toy briefly with the idea of generating the 40kHz carrier wave separately, and just gating it to the same type of signal I used for the HDMI switch. Supposedly, 40kHz generators are easy, but at least for the circuits I found at first glance, it requires a part (640kHz resonator) that is nearly impossible to find in 2020. Probably fell out of use. But as it turn out it wouldn’t have helped.

Instead, I took another Feather. Since I ran out of M4, except for the one I hardwired already an IR LED to, I instead pulled up the nRF52840 that I bought and barely played with. This should have been plenty capable to give me a clean 40kHz signal and it indeed was.

At that point I noticed another problem, though: I totally screwed up the adapter board. In my Feather M4, the IR LED was connected directly between 3V and the transistor switching it. A bit out of spec, but not uncommon given that it’s flashed for very brief impulses. On the other hand when I designed the adapter, I connected it to the 5V rail. Oops, that’s not what I was meant to be doing! And I did indeed burn out the IR LED with it. So I had to solder a new one on the cable.

Once I fixed that, I found myself hitting another issue: I could now turn on and off the TV with my app, but the switch stopped responding to commands either from the app or from the original remote! Another round of Saleae (that’s probably one of my favourite tools — yes I splurged when I bought it, but it’s turning out to be an awesome tool to have around, after all), and I found that the signal line was being held low — because the output pin is stuck high…

I have not tried debugging this further yet — I can probably reproduce this without my whole TV setup, so I should do that soonish. It seems like opening both lines for PWM output causes some conflicts, and one or the other end up not actually working. What I solved this with was only allowing one command before restarting the Feather. It meant taking longer to complete the commands, but it allowed me to continue with my life without further pain.

One small note here: since I wasn’t sure how Flask concurrency would interact with accessing a serial port, I decided to try something a bit out of the ordinary, and set up the access to the Feather via an Actor using pykka. It basically means leaving one thread to have direct access to the serial port, and queue commands as messages to it. It seems to be working fine.

Wrapping It All Up

Once the app was able to send arbitrary commands to the TV via infrared, as well as changing the input of the HDMI, I extended the Home Assistant integration to include the TV as a “media_player” entity as well. The commands I implemented were Power On and Off (discrete, rather than toggle, which means I can send a “Power On” to the TV when it’s already on and not bother it), and discrete source selection for the three sources we actually use (HDMI switch, Receiver, Commodore 64). There would be a lot more commands I could theoretically send, including volume control, but I can already access those via the receiver, so there’s no good reason to.

After that it was a matter of scripting some more complicated acts: direct selection of Portal, Chromecast, Kodi, and Nintendo Switch (which are the four things we use the most). This was easy at that point: turn on the TV (whether it was on or not), select the right input on either the receiver or the switch, then select the right input ion the TV. The reason why the order seems a bit strange is that it takes a few seconds for the TV to receive commands after turning on, but by doing it this way we can switch between Chromecast and Portal, or Nintendo Switch and Kodi, in pretty much no time.

And after that worked, we decided the $5/month to Nabu Casa were worth it, because that allows us to ask Alexa or Google Assistant to select the input for us, too.

Eventually, this lead me to replace Google’s “Turn off the TV” command in our nightly routine to trigger a Home Assistant script, too. Previously, it would issue the command to the Chromecast, routing through the whole Google cloud services between the device that took the request and the Chromecast. And then the Chromecast would be sending the CEC command to power off… except that it wouldn’t reach the receiver, which would stay on for another two hours until it finally decided it was time to turn off.

With the new setup, Google is triggering the Home Assistant script, and appears to do that asynchronously. Then Home Assistant sends the request to my app, that then sends it to the Feather, that sends the power off to the TV… which is also read by the receiver. I didn’t even need to send the power off command to the receiver itself!

All in all, the setup is satisfying.

What remains to be done is to try exposing a “Media Player” to Google Home, that is not actually any of the three “media_player” entities I have, but is a composite of them. That way, I could actually just expose the different input trees as discrete inputs to Google, and include the whole play, pause, and volume control that is currently missing from the voice controls. But that can wait.

Instead, I should probably get going at designing a new board to replace the breadboard mess I’m using right now. It’s hidden away enough that it’s not in our face (unlike the Birch Books experiments), but I would still like having a more… clean setup. And speaking of that, I really would love if someone already contributed an Adafruit Feather component for EAGLE, providing the space for soldering in the headers, but keeping the design referencing the actual lines as defined in it.

Birch Books With ATmega48

I last wrote about the Trinket M0 version of Birch Books, which is using CircuitPython and uses a much simplified code structure, while relying on one of two possible I/O expanders. But I felt a bit of a cheat to implement this one with a Trinket M0, which is a very powerful MCU to use for something that should be as simple as to keep a few lights on in a LEGO set.

So, as suggested by a colleague, I looked into the ATmega48, which is a close relative to the ATmega328 that Arduino is based on. Actually, these two versions are pin- and source-compatible so while I have now implemented it with the 48, the 328 (which is more commonly available) should work the same way.

The ATmega48 option is an interesting compromise between the 8052 and the Trinket M0. It’s an 8-bit microcontroller, and it requires to write low-level C to program, but at the same time it doesn’t require any passive components (as long as you don’t need more than 8MHz, and you didn’t make the mistake I made about having an active-high RST button). It turned out that the suggestion was a very good one, and the ATmega48 does indeed seem to fit this project very nicely.

Programming the ATmega48 can be done via 6-pin connectors that need to provide power, SPI bus, and the RST line. This can be achieved with Adafruit’s FT232H board: you connect the SPI lines just as described on the breakout, and use D3 for the RST line, then you just tell avrdude to use the UM232H variant. No other configuration is needed. I actually plan on crimping my own harness for this later, so I don’t need to use a breadboard all the time.

The other main difference between the ATmega48 and the 8052 is that the former has nearly half the pins, but it still has enough I/Os (14 outputs, 2 inputs) for Birch Books that there is no need to add an expander. There’s a few more I/O lines as well, but they are “overcommited” and by default configured to have different meanings. For instance the external clock pins can be re-configured to be additional I/O lines, and even the RST line can be disabled by fuses to gain one more I/O.

Let’s talk about the clock! The ATmega48 is sent out from factory with the fuses configured for a 1MHz clock, but can be configured up to 8MHz. Alternatively it can use an external clock and run up to… I forgot, and the point being I don’t need any of that. I did order 100 pieces of 12MHz crystals just to have them at hand, when I made an order from LCSC, but for this project, the 1MHz clock speed is plenty. So I left it configured the way it came from factory, and that works perfectly fine.

The code ended up being very similar to the 8052 version of course: it initializes the ports, sets up a timer input, and in the loop checks for the test button and otherwise keep updating the port state depending on the calculated virtual hour. While the ATmega48 would have allowed for more fine-grained controls on the timing, and thus allowed me to use the original planned scheduled, I decided to actually keep the same 1/16th of a second “tick” timer, and the same formulas to calculate the schedule.

Another thing allowed by the ATmega48, or rather the fact by the AVR-GCC, would be making the code more modular. As I ranted before, SDCC is not the most aggressive compiler, when it comes to optimizing around functions, and that’s why instead of having more structured code I ended up with a lot of spaghetti. To make it easier to compare and maintain the code together, I decided not to refactor the spaghetti in the AVR version, but I think that’s okay. The simplification around the timer handling is already a huge step forward anyway.

On the logic side, I decided to implement a one second power-on self-test though that turns on all of the lights. This was meant not just to easily test the code while I had it on my desk, but also just as a general “healthiness check” for the LEDs. I backported it to the CircuitPython code while I was at it, as well. And I removed the “knuckle pattern strobing” (which turned on every other LED alternatively), because it doesn’t work quite right or well on the real LEGO set (where two rooms use three LED lines tied together), simplifying the test logic.

The other thing that I found out by using the ATmega48 is that I do have some difference in the current limiting between the Trinket M0 and the other two controllers. When I installed the Trinket M0-based controller, the lights in the bookstore were much brighter, and even if that looked nice, it had me worried, because I knew that the LEDs I’ve been using tend to brown out quickly if they are powered at maximum current. I managed to pretty much kill one LED by keeping it running for just two days without a current-limiting resistor. And indeed, a month later, some of the LEDs are now nearly completely dark, and will need to be replaced.

Since the actuator board is the same, powered from 5V, and with 47 Ohm resistor networks for the LEDs in each case, I’m not entirely sure why this is happening. I might need to add some inline current metering to to see how that fits together. But that’s for another time.

NeoPixel Acrylic Lamp: A Few Lessons Learnt

Last month I wrote some notes about Chinese acrylic lamps, the same kind as I used successfully for my insulin reminder. As I said, I wanted to share the designs as I went along, and I do now have a public repository for what I have, although it has to be said that you shouldn’t trust it just yet: I have not managed to get this to properly turn it on. So instead, in the spirit of learning from others’ mistakes, let me show you my blooper reel.

Lesson #1: Don’t Trust The Library

A pictures of two SMT-manufactured Acrylic lamp boards side-by-side.

These were unusable.

You may remember that in the previous post i showed the “sizing test”, which was a print of the board with no components on it, which I used to make sure that the LEDs and the screw holes would align correctly in the base. Since I had to take the measurement myself I was fairly worried I would get some of the measures wrong.

The size test was sent to print before I came to the conclusion that I actually wanted to use NeoPixel LEDs, instead of simple RGB LEDs, so it was printed to host “standard” 3535 common-anode LEDs. I then changed the design for a more sparse design that used the W2812B-Mini, which is a 3535 package (meaning it’s 3.5mm by 3.5mm in size), and is compatible with the NeoPixel libraries. This meant more capacitors (although modern versions of the W2812B don’t seem to require this) but less logic around connecting these up.

As you can see from the image above, I also added space on the side to solder headers to connect past the USB-to-UART and directly to the NeoPixel row. Which was probably my best choice ever. When the boards arrived, the first thing I did was connecting the NeoPixel control pins to try to turning them on and… I discovered that nothing worked.

The answer turned out to be that I trusted the Adafruit Eagle library too much: the pinout for the 3535 variants of the W2812B (the “mini”) has been added wrong since the beginning, and an issue existed since 2018. I sent a pull request to correct the pinout, but it looks like Adafruit is not really maintaining this repository anymore.

Because of the pinout mistake, there’s no way to access the NeoPixels on any of the ten boards I had printed in this batch, and I also missed one connection on the CP2104, which meant I couldn’t even use these as bulky USB-to-UART adapters as they are. But I can put this down as experience, and not worry too much about it, since it’s still a fairly cheap build by comparison with some of the components I’ve been playing with.

Lesson #2: Datasheets Can Still Lie To You

So I ordered another set of boards with a new revision: I replaced the 3535 version with the 5050 after triple checking that the pinout would be correct — while I did have a fixed part, I thought it would be better not to even suggest using a part that has a widespread broken pinout, and I did confirm I could fit the 5050 through the aperture by then. I also decided to move some components around to make the board a bit tighter and with a more recognizable shape.

The boards arrived, without the ESP32-WROOM module on them — that’s because it’s not in the list of parts that JLCPCB can provide, despite it being sold by their “sister company” LCSC. That’s alright because I procured myself the modules separately on AliExpress (before I figured out that ordering from LCSC is actually cheaper). And I started the testing in increasing order of satisfaction: can the NeoPixel be addressed? Yes! Can the CP2104 enumerate? Yes! Does it transmit/receive over serial? Yes! Does esptool recognize the newly soldered ESP32? Yes! Does it fit properly in the base with the module on, and have space to connect the USB plug? Yes! Can I flash MicroPython on it? Well…

This is where things got annoying and took me a while to straighten out. I could flash MicroPython on the ESP32 module. The programming worked fine, and I could verify the content of the flash, but I never got the REPL prompt back over serial. What gives?

Turns out I only read part of the datasheet for the module, and not the Wiki: there’s something that is not quite obvious otherwise, and that is that GPIO0 and GPIO2 are special, and shouldn’t be used for I/O. Instead, the two of them are used to select boot mode, and enter flashing. Which is why GPIO0 is usually tied to a “BOOT” switch on the ESP32 breakout boards.

How does esptool handle these usually? By convention it expects the RTS and DTR line of the serial adapter to be connected respectively to EN (reset) and GPIO0 (boot), together with “additional circuitry” to avoid keeping the board in reset mode if hardware flow control is enabled. Of course I didn’t know this when I sent these to manufacture, and I still am not sure what that additional circuitry looks like (there’s some circuitry on SparkFun Thing Plus, but it’s not quite clear if it’s the same as Espressif is talking about).

I have seen a few schematics for breadboard-compatible modules for ESP32, but I have not really found a good “best practices to include an ESP32 module into your design”, despite looking around for a while. I really hope at least documenting what can go wrong will help someone else in the future.

Next Steps

I have a third revision design that should address the mistakes I made, although at the time of writing this blog post I still need to find the “additional circuitry”, which I might just forego and remind myself that hardware flow control with ESP32 is probably a bad idea anyway — since the lines are used for other purposes.

I also made sure this time to add reset and boot buttons, although that turned out to be a bit more of a headache just to make sure they would fit with the base. The main issue of using “classic” top-actuated buttons is that putting them on the top of the board makes it hard to find them once mounted, and putting them on the bottom risk to get pressed once I fit back the bottom of the base in. I opted for side-actuated buttons, so that they are reachable when the board is mounted in the base, and marked on the bottom of the board, the same way as the connectors are.

I’m also wondering if I should at least provide the ability to solder in the “touch button” that is already present on the base, and maybe add a socket for connecting the IR decode to reuse the remote controls that I have plenty of, now. But that might all be going for over-engineering, who knows!

Unfortunately, I don’t think I’ll be making an order of a new set of boards in the immediate future. I’ve already said this on Twitter, and it’ll deserve an eventual longer-form post, but it seems like we might be moving sooner, rather than later. And while JLCPCB and LCSC are very fast at shipping orders, I will be mostly trying not to make new orders of anything until we have a certainty of where we’ll be in a a few months. This is again a good time to put everything into a project box, and take it out when the situation feels a bit more stable.

CircuitPython-powered Birch Books

This past April, after leaving one bubble and before joining the next, I found myself deciding to work on some basic electronics, to brush up my down-to-hardware skills and learn new stuff. And since I just had assembled the Lego Bookstore (Birch Books), I thought I would improve on that by adding LEDs and controlling them with a programmed microcontroller.

I’ve been back and forth on choosing which MCU to use, settling on an 8051 clone, which was not entirely straightforward to get to work, but eventually I thought I did. Until I got the actual boards to mount it on, and found out that I couldn’t get any new code on my chips, for some reasons I still haven’t figured out. Instead I decided to take a different approach and use a higher-level programming language and higher-level board, too.

Due to the new job having different set of contribution guidelines, I had to wait a bit before making the new boards and code available, but I have described most of the plans for it in the previous blog post. The code is now dropped in the repository. And I also spent some time to tidy up some of the other boards based on the various bits and pieces I learned from the past few months of trials and error.

The CircuitPython-based implementation relies on either the MCP23016 or the MCP23017 GPIO expanders. The trick for that is that the code is looking for the former at address 32 and the latter at address 33. The pull request to support the older, clunkier ’16 expander is approved but hasn’t been merged yet at the time of writing. I briefly considered making yet another alternative board based on the MCP23018 just to add support for it to the Adafruit library, but… I’ll leave that to someone else for now.

And while I used a Feather M4 to have it working at first, I ended up building the boards to the tinier Trinket M0, which turned out not just to be a good fit for the project, but also a good way to make sure the code would work on more boards. Turns out that on the Trinket, there’s no time.time() and I had to switch to monotonic() instead, and that lead to me finding a bug in CircuitPython.

In truth, you can hotwire anything to these boards as long as they have I²C and can talk to the MCP23016/7. And while the connector on them is designed with my actuator board in mind, what it has is pretty much the whole 16 I/O lines coming from the expander. Were you looking for an easy way to connect a Trinket M0 with 16 I/O lines to a breadboard? There you go — you might just have to connect the top header as a male pin header on the bottom of the board. Making an adapter to use this with a Feather would be relatively straightforward.

The (nearly) end result is fairly pleasing:

The version you see there is not the most recent design that you’ll find in the repository, but rather the first prototype of this version of the board that, surprisingly, managed to work the first time — nearly. The actuator board needed some work after I set up the Trinket: at first it kept blinking really fast, because it kept going in and out of test mode (which turns on all the LEDs), and in and out of “fast forward” mode (I’ll get to that in a moment). The reason turned out to be that the inputs needed an explicit pull-down. Which is why the new version of the actuator has three resistors to the side of the buttons. I also learnt that I really should pay attention to the orientation of components on the design, since at first I messed up and pulled up one of the inputs constantly.

Let’s talk a moment about the “fast forward” mode. When I originally came to design the inputs for the project, I decided I would want to have a way to run the whole cycle faster, to make videos out of it without needing to use a time-lapse or something like that. This turned out to be a good idea for testing, as well. But it’s implemented in fundamentally different ways between the 8052 firmware and its CircuitPython counterpart.

In the 8052 version, the time is kept as a number of ticks increased by a timer interrupt. When the fast-forward mode is engaged, instead of adding one tick per interrupt, it adds sixty four. Which is the speed up time of the fast forward. It also means that fast-forward works similar to a fast-forward on an old cassette tape, as it speeds up time while pressed, but it starts and end with normal time.

In the CircuitPython version, the “virtual hour” (as in, which scene we’re at) is wholly dependent on the monotonic time since start up, and the fast forward mode just changes the divisor between real time and virtual time. Which means that pressing fast-forward completely changes the virtual time. Oops. I couldn’t find a decent way around that that would have made the firmware overly complicated, so I pretty much gave up.

On the bright side, since I already have a repository for flame effect code for CircuitPython, it might be a good time for me to include the design for the fireplace, since it should be trivial to do so with the boards as they are right now!

Overall, I’m fairly happy with the result, although it feels less “self-contained” than I originally planned. I was honestly afraid at first that the Trinket M0 would be more power-hungry than the STC89, but it might be the opposite. Indeed I’m now having some issues with at least one one of the LEDs that I put in the bookstore being “burned down” — I fear my current limiting calculation was too tied to the STC89, as noticed when the ground floor LEDs turned out to be much brighter than with the breadboard. I need to make sure I got my numbers right — and if I need to, I have some new, fancier LEDs I can install, although those were rated for 12V, in which case I might need some rethinking of the whole setup.

Now to wrap this up a few words about this whole experience: I have worked on hardware before, but never down to this level. I have worked on firmware for very tiny systems that go into HVAC control planes, and for bigger systems built with COTS components but treated and sold as embedded solutions. But the closest to anything like this for me has been high school, which is over fifteen years ago!

This journey has been a learning experience, and I want to thank the folks over on the Adafruit Discord server. I also welcome any comment and critique on the code or the board designs — the latter particularly because I really have no idea what I was doing, and I had to improvise.

I also want to keep talking about mistakes as I make them, and even wonder out loud when I think I did something wrong, for two reasons: first of all, as I just said, I welcome more critiques of my thought process — if I’m going around something the wrong way, please let me know. The second reason is a bit more “squishy”: while making these mistakes is a sure way to learn more, they are not cheap mistakes. Ordering another set of five prototypes is around £5, but then you need to batch a few and spread over the shipping costs, plus the “bill of materials” tend to add up, even if the single components are just a few cents. I hope I’ll be saving someone else’s money when they look around to “how did someone else do this for this stuff?”

Investigating Chinese Acrylic Lamps

A couple of months ago I built an insulin reminder light, roughly hacking around what I would call an acrylic lamp. The name being a reference to the transparent acrylic (or is it polycarbonate?) shape that you fit on top, and that lights up with the LEDs it’s resting on top. They are totally not a new thing, and even Techmoan looked at them three years ago. The relatively simple board inside looked fairly easy to hack around, and I thought it would make a good hack project to look more into them.

They are also not particularly expensive. You can go on AliExpress and get them for a few bucks each with so many different shape designs. There’s different “bases” to choose from, too — the one I hacked the Pikachu on was a fairly simple design with a translucent base, and no remote control, although the board clearly showed space for a TSOP-style infrared decoder. So I ended up ordering four different variants — although all of them without remotes because that part I didn’t particularly care for: one translucent base, one black base with no special features, one with two-colour shapes and LEDs, one one self-changing LEDs with mains power only.

While waiting for those to turn up, I also found a decent deal on Amazon on four bases without the acrylic shapes on them for about £6 each. I took a punt and ordered them, which turned out to be a better deal than expected.

These bases appear to use the same board design, and the same remote control (although they shipped four remotes, too!), and you can see an image of it on the right. This is pretty much the same logic on the board as the one I hacked for my insulin reminder, although it has slightly different LEDs, which are not common anode in the package, but are still wired in a common-anode configuration.

For both the boards, the schema above is as good a reversing as I managed on my own. I did it on the white board, so there might be some differences in the older green one, particularly in the number of capacitors, but all of that is not important for what I’m getting to right now. I shortened the array to just four LEDs to show, but this goes on for all of the others too. The chip is definitely not a Microchip one, but it fits the pinout, so I kept that one, similarly to what I did for the fake candle. Although in this case there’s no crystal on the board, which suggests this is a different chip.

I kind of expected that all the remaining boards would be variation on the same idea, except for the multi-color one, but I was surprised to figure out that only two of them shared the same board design (but took different approaches as to how to connect the IR decoder — oh yeah, I didn’t select any of the remote-controlled lamps, but two of them came with IR decoderes anyway!)

The first difference is due to the base itself: there’s at least two types of board that relate to where the opening for the microUSB port is in relation to the LEDs: either D-shaped (connector inline with the LEDs) or T shaped (connector perpendicular to the LEDs). Another difference is in the placement of the IR decoder: on most of the bases, it’s at 90° from the plug, but in at least one of them it’s direct opposite.

Speaking of bases, the one that was the most different was the two-colours base: it’s quite smaller in size, and round with a smooth finish, and the board was proper D shaped and… different. While the LEDs were still common-anode and appeared wired together, each appears to have its own paired resistor (or two!), and the board itself is double-sided! That was a surprise! It also is changing the basic design quite a bit more than I expected, including only having one Zener, and powering up the microcontroller directly over 4.5V instead of using a 3V regulator.

It also lacks the transistor configuration that you’d find on the other models, which shouldn’t surprise, given how it needs to drive more than the usual three channels. Which actually had me wonder: how does it drive two sets of RGB LEDs with an 8-pin microcontroller? Theoretically, if you don’t have any inputs at all, you could do it: VDD and VSS take two pins, each set of LEDs take three pins for the three colour channels. But this board is designed to take an IR decoder for a remote control, which is an input, and it comes with a “button” (or rather, a piece of metal you can ground with your finger), which is another input. That means you only have four lines you can toggle!

At first I thought that the answer was to be found on the other six-pin chip on the lift, but turns out that’s not the case. That one is marked 8223LC and that appears to correspond to a “touch controller” Shouding SD8223L and is related to the metal circlet that all of these bases use as input.

Instead, the answer became apparent when using the multimeter in continuity mode: since it provides a tiny bit of current, you can turn on LEDs by pointing them between anode and cathode of the diode. Since the RGB cathode on the single LED package are already marked on the board, that’s also not difficult to do, and while doing that I found their trick: the Blue cathods are common to all 10 LEDs, they are not separate for outer and inner groups, and more interestingly the Green cathodes are shorted to the anodes for the inner four LEDs — that means that only the outer LEDs have the full spectrum of colours available, and the only colour combination that make the two groups independent is Green/Red.

So why am I this interested in these particular lamps? Well, they seem to be a pretty decent candidate to do some “labor of love” hack – as bigclive would call it – with making them “Internet of Things” enabled: there’s enough space to fit an ESP32 inside, and with the right stuff you should be able to create a lamp that is ESPHome compatible — or run MicroPython on it, either to reimplement the insulin reminder logic, or something else entirely!

A size test print of my custom designed PCB.

Indeed, after taking a few measurement, I decided to try my hand at designing a replacement board that fits the most bases I have: a D-shaped board, with the inline microUSB, has just enough space to put an ESP32 module on it, while keeping the components on the same side of the board like in the original models. And while the ESP32 would have enough output lines to control at least the two group of LEDs without cheating, it wouldn’t have enough to address normal RGB LEDs individually… but that doesn’t need to stop a labor of love hack (or an art project): Adafruit NeoPixel are pretty much the same shape and size, and while they are a bit more expensive than the regular RGB LEDs they can be individually addressed easily.

Once I have working designs and code, I’ll be sharing, particularly in the hopes that others can improve on them. I have zero designing skills when it comes to graphics or 3D designing, but if I could, I would probably get to design my own base as well as the board: with the exception of the translucent ones, the bases are otherwise some very bland black cylinders, and they waste most of the space to allow 3×AAA batteries (which I don’t think would last for any amount of time). Instead, a 3D printed base, with hooks to hold it onto a wall (or a door) and a microUSB-charged rechargeable battery, would be a lovely replacement for the original ones. And if we have open design for the board, there’s pretty much no need to order and hope for a compatible base to arrive.