ESPhome

Windows on Home Assistant

Not to be confused with Home Assistant on Windows.

Context

You may remember that I spent a number of weeks in the past months reverse engineering my air conditioning controller, so that both me and my wife could control it via our Home Assistant instance (and the connected voice assistants). One of the latest reasons added onto why it was worth embarking into that particular journey was well captured by Alec of Technology Connections in his video on home energy storage — while the video was released after I already made up my mind on the topic, he had shared the idea in passing in other videos and I did pick up on it.

Basically, having a way to configure ranges of acceptable temperatures for different times of day allowed us to keep the flat cooler even during the last British heat wave (not the harshest unfortunately, as I wasn’t done by then!) — when a lot of neighbours complained about their (identical) AC being unable to keep up with the heat. The “trick” for the most part has been asking the aircon to keep the temperature below 28°C even in the night. When it crossed over that threshold at six in the morning, the aircon just turned on in living room as well as my home office to keep it under control. It was glorious.

But then it got me thinking. When the heat wave hit really really harsh, part of the problem was the “winter garden” (balcony, you may call it), which is basically a “buffer room” between the flat and the outside wall of glass. Since we live fairly up high, and face East, in the morning the temperature of the winter garden has been particularly surprising. Going over 30°C becomes a daily occurrence in the summer, with 38°C happening multiple workdays — what I did not expect was that on the hottest weekend, the thermometers would reach their reporting limit, as they cap at 50°C! From squinting at the curve, my guess is that we hit the 56°C in our winter garden, and that scared me more than a little bit.

Now, the flat is supposed to have a way out: the winter garden has two vents at either end, a “Juliette balcony” (which basically means a door to a dozen-or-so floors fall, only prevented by a single pane of glass), and a pantograph window that is supposed to be operated through an electric keypad in the balcony itself. When we moved in, the only ventilation option we had were the two vents: the Juliette balcony door was seemingly locked, and the electric window refused to open.

Escalating the problem with the electric window to our landlord, who then escalated to building management, and building company, didn’t seem to make progress – eventually we had the window working for about a week, then it stopped again – so before the heatwave I thankfully asked our landlord for permission to call in a locksmith to open the Juliette balcony. Turns out, the door was not locked, but rather the lock itself was broken — either because it was seized and it required a lot of force to even open, or because it “cooked” over the years without being moved. But in either case, after a lot of money (for our landlord) later, we could open the door. But it was scary, because it’s a giant hole in the external wall, many floors up, with basically nothing stopping the stuff in your pocket from flying off if you turn too fast.

And as it turns out, we didn’t want to leave that Juliette door open, with all the connected risks, while we were out. And we didn’t want to be caught at home by the harshest heatwave London experienced in the past… forever? Thus, it pretty much cooked.

So I kept he wishlist item of having the electric window work with Home Assistant, being able not only to open and close it remotely, but having some automation making sure it would open if the temperature would reach certain levels. That wishlist item became closer to reality last month, as I ended up taking matters into my (insulated) hands.

We had emptied out the boiler room, to take some pictures for a quote for a water softener (don’t ask, that’s going to be another blog post), and as we were cleaning I could distinctly hear a faint buzzing from the box marked “Penagraph [sic] Windows”. I decided it was due time to power the thing off and check on it. Inside I found a Windowmaster WUC102, haphazardly enclosed in its plastic cover, and… very warm. Turns out the manual instructs you not to cover the device — but it had been enclosed in a sealed equipment box all this time, great work!

Controlling The Window

Floating next to the WUC 102 I found two relay modules — the wiring of which turns out to be… well, not stable. If you move them out of the way sometimes their cables don’t contact correctly, and now the keypad on the winter garden stops working, or can only send one of the two commands (open, or close). Some of our neighbours have history of having their window stuck open over the winter, which is not great. Despite the equipment box being able to support a DIN rail, and the relay modules being rail-installable, the box is too small to fit all of it, and was left in one of the most horrible wiring situations I’ve ever seen in my life, and I seen many.

On the bright side, the fact that we now knew how the window was controlled meant we could at least find a way to control it remotely: the WUC102 has three inputs with different priorities, which means you don’t need to change the wiring of the already-installed keypad to add a secondary input. As long as the keypad is left in the “auto” (idle) position, the second input can request the window to open or close. And to make things easier overall, the WUC102 also provides a separate 24V supply, which you can use to power up some of their thermostats and other gadgets… or a control board.

Unfortunately, the way you control the WUC102 is a bit… uncommon, at least for me. Instead of a positive signal on an input point, or a positive, neutral, negative voltage application, the way you open or close the connected window is by grounding the open, or close, input contact. This is the reason why there are two relays in the same box: the keypad they installed appears to send a positive (24V) signal on either the open or close wire — the relays then turn this signal into the right grounding of one or the other input.

After the whole HVAC journey, I had a moment of terror at the idea of having to build my own board again. I have a vague understanding of the work involved in it, namely two relays and an ESP32, but I really wouldn’t want to have to wire a custom board to the actual building. So I started looking around for relay control boards that you can use with Home Assistant.

When you start looking around, you mostly find people talking about Shelly or Sonoff. These are generally well supported by Home Assistant either with their original firmware or with ESPHome, but it turned out to be a bit of a challenge to find what I was looking for. The main problem is that the majority of their offering are solving a fairly usual set of problems: lights, fans, boilers, and so on. And as such, they can simplify everybody’s life by using “wet contacts” relays — relays that are switching the same line they’re powered by (usually, on the neutral contact).

Since the WUC102 requires you to ground the inputs, you need “dry contact” relays, that are basically electrically actuated single-pole switches. You wouldn’t think this would be hard to find, and indeed it isn’t if you’re looking for a single relay: the Shelly 1 is essentially it, and it has an exposed serial port to make it easy to flash ESPHome onto it, without much faffing around. Unfortunately, there’s no equivalent dry-contacts Shelly 2 — the Shelly 2.5 has wet contacts, and pretty much everything else Shelly appears to have is specific to lights or similar use cases.

Similarly, when you look at Sonoff, their DUALR2 has wet contacts. And if you even look outside of those two brands, you’ll find that pretty much every dual-relay board, whether it is WiFi or ZigBee, does not come in dry contacts mode. I have no idea why.

Sonoff does make one model that uses dry contacts: the 4CHPROR3, but do note that only the PRO version has dry contacts, and the option to supply DC power, and at first I didn’t think it would be the right choice for me due to the power supply. Since I don’t have a 220V connection available at hand, I would need to power the board from the 24V supply available from the WUC102 — and while the 4CHPROR3 does support DC input, the specifications and the silkscreen says that it supports a maximum of 23V — one volt lower than I have available.

So next I found this MHCOZY branded board which I do not recommend. This turned out to be clearly a SONOFF clone, which meant in theory it can run ESPHome, as it follows the near exact same ITEAD plans. Unfortunately it didn’t test-point the serial port, and given the very thin spacing of the pins, I failed at soldering cables that would give me a stable serial port, even after removing one relay to get a bit more space.

If you’re curious about this particular board, though, you may want to take a look at the video below, which was me Twitch-streaming while trying to understand how the board was wired in. In a poor man’s Bigclive, you can see me poking around with a multimeter while figuring out how the ESP module was wired up.

Originally I intended to solder the serial port and come back with the flashing and configuration of ESPHome, but I gave up after desoldering one of the relays, and then figuring out I had no idea how to set the ESP into boot mode so that esptool could speak to it.

Back to the Sonoff 4CHPROR3

So I eventually went for the Sonoff 4CHPROR3, which is comparatively “more expensive” by a tenner, but if I did go for it it would have saved me a lot of grief. While the Sonoff boards are not as hacking-friendly as the Shelly, with their exposed serial ports, it was just a matter of soldering five pins on the board (well, four technically was all it needed, but might as well solder all of them at once), that expose the serial port and power.

A few jumper cables later on a UART adapter, and you can put ESPHome on one of them. The project website even comes with a recipe for it! Note that the the ESPHome recipe states it’s for the R2, but it works fine with the R3 as well, I don’t think they changed the way the pins are presented, which would have been annoying for them just as much as for us who prefer our own firmware controls.

But not all of it is a clear winner, so let’s talk a bit more about the problems with using the 4CHPROR3…

First of all, while the PRO versions do support DC input, it does appear something is off with the range of voltages. In the Amazon specification page as well as in their documentation, it says that the DC input voltage is 9 to 23V, which is why I said earlier I set it aside from the options at first. This gets more confusing because if you look at the Amazon picture of the product, the silkscreen clearly reads «DC Input: 9-24V» in two places. I guessed that 1V wouldn’t be that much of a difference, and decided to test it anyway on the bench supply first.

The other problem is that, unlike the MHCOZY clone above, the DC input is not available as a screw terminal! It is instead on a barrel jack plug, center-positive (as in, the most common one, that is used by studio lights, TS-100, Western Digital drives, and so on. I actually have a lot of different options to supply those, because I found myself having so many different devices supplied by the same barrel jack, that I ended up ordering a few “squids” that can power multiple devices off the same brick.

To solve that part I ended up ordering a bunch of screw terminal adapters, because turns out that those have been handy in other situations before, particularly to allow me to supply random voltages through the bench supply, so it sounded like a good thing to stash up on.

Did the different voltage make a difference? I am not entirely sure, because one thing definitely felt awkward, but I’m not sure if it’s related to the voltage, ESPHome bring-up that slightly differs between R2 and R3, or something else entirely: the fourth relay triggers for a brief moment at a cold startup. Besides that, it appears the board is working fine, and not overheating or shorting, at 24V at all. Certainly it would have felt less awkward if the polarity of the plug was explicit on the silkscreen, and if they wouldn’t put a voltage rating that excludes a very common voltage in use for home automation!

A final note: exactly like for the MHCOZY above, the Sonoff 4CHPROR3 includes 433MHz remote control support. And exactly like the MHCOZY, this is totally independent of the ESP chip on board, which means you cannot disable it even when running your own custom ESPHome firmware, at least not from the point of view of being able to distinguish the 433MHz “remote presses” from the pressing of the local inputs. What to do with those input, though, is up to your firmware, so you don’t need to worry as much as crossed signals to the WUC102 as long as you configure your firmware to avoid that situation.

Wiring And Configuration

I ended up drawing the following diagram (for the electrician installing this officially), which shows how to wire up the two devices:

Wiring diagram for the Sonoff 4CHPROR3 to WindowMaster WUC102

According to this diagram and the WUC 102 manual, R1 is “open” and R2 is “close” signal — but in reality, this does not appear correct, and indeed they are inverted, but it was definitely easier to just fix it in the firmware configuration for that. I don’t know if this is specific to the way our unit is wired to the actual window (it could be, lots of residents of the building have complained about the wiring being a problem), so if someone else has a WUC102 and finds that they are not inverted, please let me know. The good thing is, the WUC102 doesn’t care if you send a “close” signal while the window is closed or vice-versa.

Now, for the configuration things are fairly easy thanks to ESPHome’s support for covers, including documentation on how to send a timed input to one. The configuration I’m using is the following:

substitutions:
  node_name: "sonoff-wuc102"
  node_id: "sonoff-wuc102"
  device_verbose_name: "Sonoff WUC102"

packages:
  ha_wifi: !include common/ha-wifi.yaml

esphome:
  name: ${node_name}
  platform: ESP8266
  board: esp01_1m

# Device Specific Config
status_led:
  pin:
    number: GPIO13
    inverted: True

binary_sensor:
  - platform: gpio
    id: button_1
    pin:
      number: GPIO0
      mode: INPUT_PULLUP
      inverted: True
    on_press:
      - cover.open: "pantograph_window"

  - platform: gpio
    id: button_2
    pin:
      number: GPIO9
      mode: INPUT_PULLUP
      inverted: True
    on_press:
      - cover.close: "pantograph_window"

  - platform: gpio
    id: button_3
    pin:
      number: GPIO10
      mode: INPUT_PULLUP
      inverted: True
  - platform: gpio
    id: button_4
    pin:
      number: GPIO14
      mode: INPUT_PULLUP
      inverted: True

switch:
  - platform: gpio
    pin: GPIO12
    id: "relay_1"
  - platform: gpio
    pin: GPIO5
    id: "relay_2"
  - platform: gpio
    pin: GPIO4
    id: "relay_3"
  - platform: gpio
    pin: GPIO15
    id: "relay_4"

cover:
  - platform: template
    id: "pantograph_window"
    name: "Pantograph Window"
    open_action:
      # Cancel any previous action
      - switch.turn_off: relay_1
      # Turn the OPEN switch on briefly
      - switch.turn_on: relay_2
      - delay: 60s
      - switch.turn_off: relay_2
    close_action:
      - switch.turn_off: relay_2
      - switch.turn_on: relay_1
      - delay: 60s
      - switch.turn_off: relay_1
    stop_action:
      - switch.turn_off: relay_1
      - switch.turn_off: relay_2
    optimistic: true
    assumed_state: true

You can see that I’ve basically copied the configuration that is provided in the documentation, with the difference of changing the timing to 60s instead of the 1s — to be honest, I think this worked fine with 30s, but it just makes it easier when you look at the actual board in the boiler room.

The first two buttons (which control the closing and opening relays in the default configuration) have been reconfigured to send the open/close action instead, that way there’s no risk of turning on both relays at the same time. As far as I can tell, you should be able to do the same with the original firmware and the Sonoff app in the “Interlocked” mode, but I much prefer a firmware I can control locally.

I left in the configuration the additional relays and the buttons, although they are not configured to do anything. I guess one day they may have good use for something — I would love if the boiler was using a controllable relay, rather than the stupid timer it uses now, but that’s not going to happen while we rent, I think. For now, since they don’t appear to cost me anything to keep there, they are in the configuration. I guess I could just comment them out until I need them and see if the firmware becomes smaller, but does it matter?

Automation and Results

Once all of this was done, the first obvious result was to have a reliable way to open the window. Remember that since moving in in October, we had no way to to open the window until September! The keypad only works for exactly one week in April, and that was about it. So even without any automation, and even if we only counted on the buttons on the actual Sonoff board, we already had a significant quality of life improvement.

The next level of quality of life improvement was the ability to open/close the window without going into the 40°C (and over) winter garden (although it has not reached those temperatures since we installed the Sonoff) — or the closer-to-0°C if we need to vent out humidity, which was the problem we (and a number of other residents of the building) face in the winter: bad insulation between winter gardens means we end up with fairly humid spaces, that can grow mold (this is my original reason to set up CGG1 monitoring in there!). The best solution is to open vents/windows for a little time each day to reduce the built up humidity, but it’s really inconvenient to do that when it’s freaking cold and you just woke up or are about to go to bed.

CGG1 sensors reporting the winter garden temperature, and the state of the pantograph window.

And all of this indeed was delivered by just installing the Sonoff. We could have gotten most of that already with the MHCOZY with the eWeLink app, to be honest — but that was not where I wanted to stop. Instead, I wanted to make it better, greener, and cheaper to keep the flat cool.

To understand what I mean, you need to know that the temperature monitoring on the winter garden showed that, any day that is even barely sunny, it gets warm extremely quickly, with a near-vertical graph (like shown in the picture here), but it can start dropping or at least not raising further, the moment you open one of the windows — we have noted that during the heatwave when opening the Juliette balcony window.

Now that we’re closer to the equinox, it takes some time for the temperature to reach annoying levels, so we could just open the window as we wake up… but closer to the summer solstice, the temperature reaches 30°C when we’re still fast asleep (neither of us is an early bird, and working from home didn’t make it any better).

So I ended up setting up an automation rule that, if the temperature in the winter garden stays above 24°C for more than 10 minutes, it should just open the window for us. And as you can see from the graph above, it works like a charm: the temperature climbs up fast, reaching 24°C, but then drops down to a more reasonable 21°C fairly quickly.

You may wonder what’s that graph line called “Cupboard” in there — that’s a sensor attached inside a black cupboard that we put in the winter garden. The built up of humidity made it less likely we would want to use it for storing papers and board games as we originally intended, but we wanted to confirm just how bad it would be. Turns out, it’s not as bad, but definitely not a good place to store humidity-sensitive content.

This automation is still not perfect. Starting with the fact that I did not configure it yet to close the window ever. That one is left up to us for now, because I don’t have enough datapoints to know when it’s a good time to close it without losing the benefit. I think at the very least I should make sure that if it starts raining it closes by itself — right now it does check the reported weather to decide if it’s safe to open (being on the flight path to Heathrow does make that reliable enough, I find).

It also doesn’t open if we’re traveling — there’s a “big red button” throughout all our automations that turns most of them off if we’re not going to be in overnight. The reason for that is that most of them are meant to keep running if we’re out during the day, because we still want to come home to a comfortable apartment. For the HVAC, we just have different ranges, and I think I’ll need something similar for the window. A “passive thermostat” would probably also work for this, to be honest.

This is unlikely going to be a problem until summer anyway, not just because we wouldn’t be traveling, but also the only case in which I would want to open the window if we were traveling would be if the temperature would reach 45°C or something along those lines. Because in that case, it would be a risk to the contents of the flat.

Final Thoughts

I’m surprised how hard it was to find a workable solution for this — particularly one that didn’t come with more features than you need: ITEAD-based designs (like Sonoff) don’t appear to have dry contacts control boards that don’t also include the 433MHz commands. And Shelly, that appears to be a lot more hacker-friendly, does not appear to have any dry contacts options for more than one relay.

If you look around at all the various options, by the way, you’ll find that all the instructions to flash ESPHome have huge, scary warning to never try to flash these control boards while installed. The reason for that is that none of them appear to provide isolated serial ports pins, which means you have the risk of getting a full line level AC on the pins, which is… possibly deadly.

Maybe Nabu Casa can start selling ready-to-flash control boards that don’t penny pinch of safety hacking, it would be definitely something I’d be interested to pick up on myself.

It would also be interesting to see WindowMaster providing an ESPHome-flashable option to install with their WUC units — even though the WUC102 is discontinued, it does appear the protocol to speak with the new ones haven’t changed, as the RJ port on the device is still present.

But for now, I’m just satisfied we can open and close the window without the hassles involved in the previous options.

Reverse Engineering an LG Aircon Control Panel — Fit It All Together

Since we moved to this flat, one of my wishlist item was to have a way to control the HVAC (heat, ventilation, and air conditioning) without having to get out of bed, or go to a different room. A few months ago, I started on a relatively long journey of reverse engineering the protocol that the panel used so that I could build my own controller, and I’ve been documenting pretty much every step along the way, either on Twitter, this blog, or Twitch. As I type this, while I can’t say that the project is all done and dusted, I’ve at least managed to get to the point where I’m reaping the benefits of this journey.

You may remember that at the end of the previous chapter in this saga I was looking at controlling the actual HVAC with a Python command line tool. I built that on stream, to begin with, and while I didn’t manage to make it work the way I was hoping (with a curses-based UI), I did manage to get something that worked to emulate both the panel and (to a minor point) the HVAC itself.

I actually used this “in production” (that is, to control the air conditioning in my home office) for a little over a week. The way I was using it was through a Beagle Bone Black which I had laying around for a long while – I regret not signing up for the RISC-V preview, but honestly I wouldn’t have had time – connected through an USB-to-UART adapter (a CH340 based one, because they can actually do 104bps!), and a breadboarded TLIN1206 on a SOP-8-to-DIP adapter. A haphazard and janky setup if you’d ever seen one, but it was controllable by phone… with JuiceSSH and Tailscale.

While absolutely not ergonomic, this setup still allowed me to gain the experience needed with the protocol, in order to send the PCB design to manufacture. I did that about a week in, and as usual, JLCPCB‘s turnaround is phenomenally fast, and by the following Monday I had my boards.

The design is pretty much the same as the one I spoke about in part two, with two bus transceiver blocks, a 3.3V DC-DC converter, the quad-NAND to make sure the code cannot send data to the bus if the physical switch is turned to “disable”.

While the design proven it needs a little bit more work to be optimal, it’s a great starting point because it just works fine. And while I did originally plan to have this support the original panel with the switch turning on “pass-through mode”, I decided that for the moment, this is not needed, nor desired.

My original intention was to allow the physical switch to just prevent the custom controller from talking to the HVAC engine, and let the original panel take over. Unfortunately this does not work: the panel is not entirely stateful, but there is a bit in the command packet that says “Listen to me, I’m changing configuration” — and if the configuration in the panel and the HVAC don’t match when that bit is not set, an error state is introduced.

This basically means I wouldn’t be able to seamlessly switch between the old panel and my custom controller. Instead what I’m going to be doing if I need to is to first flip the switch to disable the custom logic, and then connect the panel to the secondary bus. That way it would initialize directly on the bus. If I do need to re-design this board, I’m going to make the switch more useful, and add a power disconnection so that it can be all connected without power on at all.

There was another reason why I originally planned to support the original control panel: it reports a temperature. I thought that I would be able to use that temperature as a “default” sensor, while still allowing me to change the source of the temperature at the time of configuration. But the panel’s temperature sensor is pretty much terrible, and it’s only able to measure 16°C~30°C, plus it’s easily fooled by… a lamp. So not exactly something you can call reliable for this use, and not something I would care to add to my monitoring.

To my own astonishment, my first attempt at soldering the full board was successful — two out of three of the boards work like a charm, the third one is a bit iffy, but it might be a not perfect component into it. I’ll have to see, but also I don’t want to make a new respin now that even some of the components I need are getting harder (and more expensive) to find. Take the JST ZR connection: in the picture above you an see it white rather than cream — that’s because it’s not an official JST part but an AliExpress clone that fits the same footprint, and I could actually buy.

Custom ESPHome Climate

Once I had the board, I rigged up a quick test bed on my desk with a breadboard and another CH340 adapter (I have a few around, they are fairly cheap and fairly versatile), and started off to complete the Custom Climate component.

Since I started this project, the ESPHome Climate API actually changed a couple of times, particularly as Nabu Casa is now sponsoring the project, and its development moved to a monthly release kind of deal. Somehow the Climate components were among those that most required work.

But the end result is that the API was clearer, and actually easier to implement, by the time I had this ready. So I wrote down the code to generate the six bytes packets I needed to send, and ran it against the emulator… and it seemed to work fine. I had wired this with a test component in Home Assistant and I could easily change the mode, the temperature, and everything else just fine, and I could see in my emulator that it was getting the right data in just fine.

At that point I was electrified, and I thought it would just be a matter of putting it to the wall and see it work. You can imagine my disappointed when I called in my wife to assist me in my victory… and it didn’t work. And I was ready to detach all of it to spend the next day debugging what was going on, until I realized that I forgot to send the “settings changed” flag. I have to say that the protocol turns out to be a bit more complex than I expected it to be, and I should probably write a bit more documentation about it, not just scatter it around the (now multiple) implementations.

After that, I actually went ahead and replaced all three of the control panels with my custom ones, and connected them to Home Assistant. That turned out to be a much easier prospect than anything else we have been trying up to now: we can decide which temperature reading to use to control the room, rather than being stuck to the silly temperature sensor in the panel, and we can use the two-point set temperature in the way most modern smart thermostat can (which the panel didn’t support, despite having an “auto mode” that would turn on cooling or heating “as needed”).

The first couple of days lead to a bit of adjustments being necessary — including implementing a feature that my wife requested: when not cooling or heating, the original panel would enter “fan only” mode. Which I enjoy for myself in the office, but bothers my wife. The original panel does not have an option to turn the fan off — but I could implement that in the custom controller. This allows us to keep our thermostat on the heat-cool mode most of the time, and just make sure the range is what we actually care for.

I also made the mistake at first of not counting on hysteresis. That turned out to be a bit more annoying to implement, not in the matter of code, but in the matter of the logic behind it — but it should now be working: it means that there is more friction to change the state of the air conditioning, which means the temperature is not as constant, but it should be significantly easier to run. To be honest I was impressed by how stable the temperature was when I left it to short cycle…

Home Assistant Integration

This was probably the simplest part of the whole work! Nabu Casa is doing an awesome job at keeping the two projects integrated very well, and with the help of “packages” configuration, replicating the configuration for the three separate boards took basically no time.

The only problem I had was that I couldn’t seem to be able to flash the first ESPHome firmware onto my ESP32 devkits using the WebSerial support. I have used it multiple times in the past, particularly to update my BLE Bridge, which I just need to connect to my main workstation rather than the standalone power supply, to upgrade, but for the pristine ESP32 devkits it didn’t work out quite as well.

The UI is very similar to the one that Google Home exposes for Nest thermostats where they do support air conditioning. And indeed, with the addition of the Home Assistant Cloud service, the same UI shows up for these thermostats.

And at that point it was just a matter of configuring the expected range of operation, both for the “daytime” and for the “night scene”. Which is one of the reasons why I wanted to have thermostat that we can control with Home Assistant.

You see, some time ago I set up so that we have a routine phrase (“To the bedroom”) and Flic buttons in both the living room and the bedroom, that prepare us to get to bed: turn off the TV, subwoofer, air fresheners, lights everywhere except bedroom and bathroom, set the volume of the bedroom speaker for the relaxing night sound, and so on.

Recently, thanks to a new Dyson integration, it also been setting the humidifier to raise the humidity in the bedroom (it gets way too dry!), and turn on the night mode on the other purifiers, which has been a great way not just to make it easier not to forget things, but also to save us from leaving the humidifier running 24/7: it’s easier for us to keep it running overnight at a higher humidity, than trying to keep it up during the whole day.

Now, with the climate controls in place, we can also change the temperatures before going to bed, rather than turning it off, which is the only option we had before. And this is a big deal, because particularly for the living room, we don’t want it to get too scorching hot even if we’re not there: it’s where the food cupboards, among other things, and during the heatwave we exceeded the 30°C for a couple of hours every other day. Being able to select different ranges while we’re still sleeping gives us a bit more safety, without having to keep running the air conditioning overnight.

Features And Remote Control

Similarly, our scheduled morning routine, configured to go off together with the alarm clock, can scale back the range in my home office to something suitable for my work day (it can get warm fast with computers and stuff running, and the door closed for meetings), so I don’t have to over-run the office in the morning when having my first meeting: it starts automatically, and it welcomes me with a normal, working temperature for my first meeting.

The final point is that, since we can actually set a very wide range on the thermostats, and rely on much more accurate thermometers that are not restricted to the 16°C to 30°C range, we can leave the thermostat on, with a very wide range, when we’re not actually going to be at home. This is particularly interesting now that we might be able to travel again, at least to see families and friends — we don’t want to leave the air conditioning or heating running all the time, but we also want to have some safeguards against the temperature dropping or raising out of control. This became very clear when the CGG1 in our winter garden hit the 50°C ceiling it could report while we were out playing Pokémon Go and we couldn’t turn on the air conditioning at all — thankfully the worst result of that was that the body of one of the fake candles we have in the winter garden for ambiance melted… and now it looks even more realistic as a candle (it also damaged the moving flame part, but who cares.)

Now, the insulation of this flat is not great. In particular the home office tends to drop down close to freezing temperatures in the winter, because the window does not seal even when closed. This means I can’t easily follow Alec’s suggestion on energy storage. But having a bit more control on automation does make it easier to keep the temperature in the flat more stable. In the winter, I expect we’ll make sure that we keep a minimum temperature overnight to avoid having to force a much higher differential when we wake up, for instance.

There’s a few features that I have not yet implemented, and that I definitely should look into implementing soon. To start with, as soon as summer gives away to autumn, we’re likely going to want to be using the dehumidifier more — without turning on the heat pump, particularly in the living room as we cook and eat. Since the CGG1 provides humidity readings together with temperature, it means I can set up an automation that, if nothing else is running, turns the dehumidifier on if the humidity reaches a certain set point, for instance.

There’s also two switches that I have not implemented yet, but should probably do, soon. One turns on resistive heating – and this will make sense again if you watch Alec’s video on heat pumps – while the other has to do with the Plasma Filter.

What’s a plasma filter? That’s a good question, and one that I’m not sure I have the right answer for. I know that this is something that the original control panel suggests is present in our HVAC, although I have no way to know for sure (we don’t have the manual of the actual engines). The manual for the PQRCUDS0 says that «you can use the plasma function» but also states «If the product is not compatible with the Plasma function,
it will not do the Plasma function even though the indicator is turned on.» This suggests that unlike other features like the swirl/swing, it’s not part of the feature query that the panel sends at turn on.

When googling further to look for information about LG’s plasma filter, I did find another manual, for an actual unit, rather than the control panel. Not the unit we’ve got, I think, but at least an unit. And this one has a description for the plasma functionality:

Plasma filter is a technology developed by LG to get rid of microscopic contaminants in the intake air by generating a plasma of high charge electrons. This plasma kills and destroys the contaminants completely to provide clean and hygienic air.

This is quite a bit interesting — and next to this, a video from LG refers to this as a “Plasma/Ionizer”, which pretty much suggested me that this is one of Big Clive’s favourite toys: ozone generators. Which makes sense given that one of his favourites is a Sharp Plasmacluster.

Code And Next Steps

First of all, the code and the board designs are all available on my GitHub. I originally considered making this a normal component for ESPHome, but since it relies on a very custom board, it doesn’t feel like the right thing to do. It does mean I need to manually keep in mind all the various changes in APIs, but hopefully that will not take too much of my time.

As I said previously, I have not actually implemented the panel bus handler — the panel will enter into error mode if it does not get an expected reply from the HVAC engine, so connecting it right now would not work at all, except if you were to disable the actual ESP32 control. I’m likely going to leverage that behaviour to test some more error handling in the future.

I would like to put a box around the board — right now it’s literally stuck to the wall with some adhesive feet, in all three rooms. And while the fixed red LED is not too annoying overnight, it is noticeable if you wake up in the middle of the night. My original idea was to find someone who can help me 3D print a box that fits on the same posts the panel fits, and provide a similar set of posts for the original panel. But it also involved me finding a way to flip the switch without taking it down the wall.

But since figuring out 3D printing with no experience is going to take a lot of investment, I am not going to take a look at that option until I’m absolutely certain I’m not changing anything in the design. And I know I would want to change the design a little bit.

First of all, I want to have a physical cut-off for the connection — since the power to run the ESP32 module comes from the same cable controlling the HVAC, the only way to turn off the power is to disconnect the cable, right now. Having a physical switch that just disconnects the power and data would just make it easier to, say, replace the devkit module.

Similar, as I said, the panel is not that useful to keep running all the time. So instead I would change the switch implementation to keep the panel off most of the time, and only power it on when disabling the ESP32. This would also save some components, since there would be no need to have the second bus transceiver and its passives.

I’m also wondering if it would make sense to have some physical feedback and access, in addition to the Home Assistant integration and the voice assistant controls: in particular I’m considering having an RGB LED on the board to tell the current action being taken (optionally, I wouldn’t want to have that in the bedroom, as it would be way too bright) together with a button to at least turn the HVAC “soft off”.

Finally, there’s a couple of optimizations that could be done to make the board a bit cheaper. One of the capacitor is ceramic, but could be replaced with a polymer one for ⅓ of the price, and the TVS diodes pair (which were actually a legacy part of the design, recommended by the MCP2021, but not in the reference designs for the TLIN1027) could be replaced with a single integrated TVS diode — it would just be “a bit” harder to solder by hand in TO-236 packages.

These are all minor though — the main cost behind the board is actually the ESP32-DEVKIT-32D that it’s designed around. It would be much cheaper to use only the ESP32 WROOM module, and not have the USB support components on the board. But I have had bad experiences with trying to integrate that in my designs, so I’m feeling a bit sceptical of going down that route — it also would mean that a botched board sacrifices the whole module (I did sacrifice two or three of those already) unless you get very good at desoldering them (or you have a desoldering station).

So most likely it will take me a few months before I actually get to the point of trying building a 3D printed cover for it. With a bit of luck by then we’ll be back in an office at least part of the time, and I can get someone to teach me how to use the 3D printer there.

Also as a final note — the final BOM for the boards suggest that building one of them costs around £25 or so, without the case. As I said there’s a few cost saving measures that I could take for the next round — though it’s questionable, because it would require me to get more components I don’t currently have. Of course the actual cost of building three of them turned out to be… significantly higher.

I think this is the part that sometimes it’s hard to explain to people who have not had this type of experience: the BOM costs are only one of the problems you need to solve — you can really screw up a project by choosing the wrong components and bringing the BOM price too high… but a low BOM cost does not make for a cheap project to finish, particularly when you’re developing it from nowhere.

In this case, if I wanted to tally up the cost of building these custom thermostat panels, I would have to, at the very least, count the multiple orders from DigiKey from which I ordered the wrong components (like the two failed attempt at using Microchip’s LIN bus transceivers rather than the TLIN, or the discrete UARTs with their own set of passives). But then there’s the cost of having all of the various tools that were needed to get this all done. Thankfully most of those tools have been used (and sometimes abused) for different projects, but they are a good metaphor for the cost of R&D that many products need — so it makes sense that what you end up buying costs more than the “simple” expense of the BOM. So keep that in mind next time you see an open source hardware device costing more than what you expect it to.

Reverse Engineering an LG Aircon Control Panel — Introduction

I like reverse engineering stuff. It’s not just the fact that it’s a nice puzzle to solve, but I enjoy the thrill of “Oh, that’s how that works.” I’m sure I’m not alone, as can be clearly seen following marcan’s Asahi Linux work, or following Foone on Twitter, or Big Clive on YouTube (and many, many others).

Sometimes, a lot more rarely, my reverse engineering is actually geared towards something I want to make use of, rather than just for the sake of finding answers — this is one of those cases. If you have been following me on Twitter or decided to watch me work on this live on Twitch, you probably already know what I’m talking about. If not, be warned that this is going to be the first part of a (possibly long) series of posts on the same topic. It turned out to be very long for a single post, and I decided to split it instead.

You see, when we moved from the last apartment, we sold our Nest smart thermostat to a friend. The new apartment has an aircon system with heat pump, rather than a “classic” heating system, which is really important as the balcony can easily reach 40°C in the mornings when the sun shines. And unlike in the US, where thermostats are pretty much standardized, Europe’s landscape of thermostats is different enough that Nest gave up, and does not support aircon systems.

Aside: I do have a bit of a rant about Nest Thermostats in Europe, but some of that might be a bit tricky to phrase for me without risking breaching confidentiality with my previous employer, which I don’t want to do. So I will leave a question here for European Nest Thermostats users: can you finally enable hot water boost with the Google Home app?

To be honest, this also kind of makes sense: in a flat that is cooled and heated with an HVAC, it makes sense to have multiple thermostats so that each room can set a different required temperature. If we’re spending the evening in the living room, what’s the point of heating up the bedroom? If I’m on vacation and not spending time in the office, why would I turn on the air conditioning? And so on.

Unfortunately what we ended up with is three thermostat units from LG, model number LG-PQRCUDS0 (provided for ease of searching and finding this blog post), which are definitely not smart, and also not convenient. These are wired, non-smart control panels, that do support features like timing, but do not provide any way to control without tapping on the screen. As far as I know, these are configured to read a temperature sensor that is not on the panel itself, but on the other hand, the placement of those sensors are a bit on the unfortunate side: in particular in the bedroom it appears located in a position that is too natural to fit a wardrobe in, making it register always a higher temperature that the room actually has.

This had been particularly annoying during the winter but it was proving to be worse during the summer: as I said the temperature in the balcony can reach 40°C in the morning, as we’re facing east and it’s a all-glass external wall. That means that the temperature inside the apartment can easily reach 30°C quite suddenly. This is not good for electronics already, but it’s doubly non-good for things like food and medicine, including insulin, which I very much depend on.

While we could just try leveraging the timer mode to turn on the AC in the morning, the complication of where the sensor is makes it very hard to judge the temperature to set it at. And since, as Alec points out on the video, the thermostat’s job is only to turn something on or off (in theory, at least)… well, there has to be an easier way.

So I embarked in this quest of reverse engineering my aircon control panel, with the intent of introducing an ESPHome-compatible add-in that would allow me to control the HVAC through Home Assistant.

Inspection

The first thing to do when setting off to reverse engineer something is to figure out what it is, whether there is any documentation for it, and whether someone else already reverse engineered it. The model number, as I said, is LG-PQRCUDS0 and LG has user and installation manuals online describing it a Delux Wired Remote Controller (together with the -B and -S variants of the part number).

Reverse image search for the panel actually seemed to struck gold at first, as this Instructables post showed exactly the same UI as mine, and included a lot of information about the protocol. But also the comments pointed to a couple of different models that seemed all similar but a bit different. So instead of going ahead and trying to build the already reversed protocol I wanted to confirm how it all worked myself.

A close up of the door behind my LG aircon control panel showing a JST ZR connector, and a yellow-red-black cable going to the wall.

The first question is going to be what the electrical “protocol” it’s using. The back of the panel has a door, that hides the inbound connection from “the wall” (that is, the actual HVAC unit), which is three wires and terminates in a JST ZR connector.

With my multimeter I could confirm that the voltage would be around 12V — but I couldn’t confirm whether it would be differential data or what else, since I’m still using an older multimeter and it doesn’t have any option to indicate there’s a signal on a wire. If someone has a good suggestion for a multimeter that does that, please leave a comment below the video in this post as I’d love to get a good one.

Now this is a good news, overall. The fact that the plug, and the cable itself, can be bought off the shelf means I don’t have have to take risky approaches, which is great, given that we’re renting, so any reverse engineering and replacement implementation needed to be non-destructive and reversible.

So I took out my Logic Pro, a very long USB 3.0 cable, and I ordered just enough components from Digikey to debug this thing. And a bench power supply — because I didn’t have a bench power supply, and given this thing needed 12V, it sounded something handy to have for this. The end result is the following:

With this connected, I used the Logic 2 software to check the voltage levels, and figure out that the yellow wire is data, while the red wire (in the middle) is 12V supply. The data turned out to, indeed, be a 104 Bd serial connection, which would make it share a lot of the information from the previous reverse engineering…

Except that something was off: what I could see on the wire was a burst of 12 bytes in a single stream, exactly once a second, which I assumed at that point to be unidirectional from the panel to the HVAC. But when trying to verify the checksum it didn’t match what the instructions on the other project suggested: sum everything, modulo 256, and xor with 0x55 (the confusing ‘U’ in the various descriptions is actually a bit pattern). So while I could figure out that the first byte seemed to include the mode of operation, and the third one appeared to include the fan speed, I couldn’t figure out for the life of me the checksum, so I thought I wouldn’t be able to send commands to the HVAC to do what I wanted.

On the other hand, in the worst case scenario I could have just replayed the commands I could record from the panel, so I decided to try my luck at drawing and ordering a PCB that would have just enough components for me to play around with.

Drawing the PCB

I’m far from being even a passable expert on electronics, but I could at least figure out some of the things I wanted from a “smart controller” unit to attach to this aircon. So I started with a list of requirements:

  • Since I wanted it to use ESPHome, it should be designed around an ESP32 module. I already attempted this once with the acrylic lamps, and I have yet to get a working board out of that. On the other hand, this time I’m much less space constrained, so I decided to go for a full DEVKIT module, one of those with already the full board of regulators, USB port and serial adapters. This turned out to be a further blessing in disguise, since the current chip shortage appears to have affected the CP2104 module I used in my previous design and I wouldn’t have been able to replicate it.
  • While I don’t expect that the HVAC power supply has been limited in power significantly (after all there’s even more deluxe WiFi enable controllers in other versions), I didn’t want to increase the load on the 12V supply significantly. Which meant I went for the more complex, but also more efficient, route of building in a buck converter to 3.3V to power up the ESP32.
  • Also, I really know that relying on my code for “enjoyment-critical” use cases can be frustrating, I wanted a physical way to hard-disconnect a possibly misbehaving automation, and go back to use the old controller, without having to fidget with cables.

With these conditions, and the assumption that the twelve bytes I was seeing were being sent directly from the controller to the HVAC, I drew and manufactured the above board. Feel free to spot the error at the top of the board, if you may.

Now, since JLCPCB turnaround is usually fairly fast, I went ahead and got that manufactured while I was still fighting with figuring out the checksum. So when the boards arrived and I populated them, I was planning on just keep changing settings to find more possible combinations of bytes to see how the checksum would behave.

And that’s when I found out I was very wrong in my assumption, and it’s possible that either the reverse engineering notes I’ve seen for other are missing a big chunk of information, or LG has so many different ways to achieve roughly the same endgame. One I powered up the panel from the bench supply, then I could see that the panel was rather only sending six bytes, rather than the twelve I expected. It’s a bidirectional communication on a single wire, a bus.

That meant going back to the literal drawing board, find the right components to implement this, and start what turned out to be a much large sidequest of complicating matters.

Flashing ESPHome on White Label Smart Plugs

You may remember that I bought a smart plug a few years ago, for our Christmas tree, and had been wondering what the heck do you use a smart plug for. Well, a few months ago I found out another use case for it that I had not thought about to that point: the subwoofer.

When I moved to Dublin, nearly eight years ago now, I bought a new TV, and an AV receiver, and a set of speakers — and with these last one, came a subwoofer. I don’t actually like subwoofer that much, or at least I didn’t, because the low-frequency sounds give (or gave) me headaches, and so while it does sound majestic to watch a good movie with the subwoofer on, I rarely do that.

On the other hand, it was still preferable to certain noise coming from the neighbours, particularly if we ended up not being in the same room as the device itself. Which meant we started using the subwoofer more — but unlike the AV receiver, that can be controlled with a network endpoint, the subwoofer has a physical switch to turn on and off… a perfect use case for setting this up with a smart plug!

Since I didn’t want to reuse the original plug for the Christmas tree (among other things, it’s stashed away with the Christmas tree), I ended up going online and ordering a new one — and I selected a TP-Link, that I thought would work just the same as my previous one… except it didn’t. Turns out TP-Link has two separate Smart Home brands, and while they share the same account management, they use different apps and have different integration support. The Christmas tree uses a Kasa-branded plug, while the one I ordered was a Tapo-branded plug — the latter does not work with Home Assistant, which was a bit annoying, while the former worked well, when I used it.

When TP-Link then also got in the news for having “fixed” a vulnerability that allowed Home Assistant to control the Kasa plugs on the local network, I got annoyed enough that I decided to do something about it. (Although it’s good to note that nowadays Home Assistant works fine with the “fixed” Kasa plugs, too.)

I asked once again help to Srdjan who has more patience than me to look up random IoT projects out there, and he suggested me the way forward… which turned out a bit more tortuous than expected, so let me try to recount it here, showing shortcuts as much as I can at the time of writing

The Hardware

Things start a bit iffy to begin with — I say that because the hardware I ended up getting myself was Gosund SP111 from Amazon, after checking reviews that they were still shipping an older version of the hardware that can be converted with tuya-convert (I’ll get back to the software in a moment). There’s plenty of warnings about the new models, but in a very similar fashion to the problem with CGG1 sensors, there’s no way to know beforehand what version of a firmware you’re going to find.

But in particular, SP111 are not Gosund engineering’s own work. Tuya is a company that build “white label” goods for many different brands, and that’s how these plugs were made. You can find them under a number of different brand names, but they all share the same firmware and, critically, the same pairing and update keys.

One of the best part about these Tuya/Gosund plugs is that they are tiny in the sense of taking very little space. They are less than half the size of my first Kasa-branded TP-Link smart plug, and even when compared with the newer Tapo-branded one they are quite slimmer. This makes for adding them easily to places that are a bit on the constrained side.

The Firmware

I did not buy these devices to run with the original firmware. Unlike the CGG1 and pretty much everything else I’m running at home, this time I went straight for the “I want to run ESPHome on them.” The reason for that was on one side, I’m annoyed and burnt by the two TP-Link devices, and from the other, the price of Hue-compatible smart plugs was annoyingly high. That would have been my default alternative.

Of the various important things to me, Home Assistant support grown from a “meh, sure” to “yeah I want that” — and in part because I did manage to set up scripts for quite a bit of tools at home through it. One of my original concerns (wanting to still be able to control most of these features by voice) is taken care of by signing up for Nabu Casa, which provides integration for both Google Assistant and Alexa, so even with the plug running a custom and maybe janky, firmware, getting it to work with the rest of the house would be a breeze.

There’s other options for open source firmware for the SP111, as well as other smart plugs. Tasmota is one of the most commonly used for this, and there’s another called ESPurna. I have not spent time investigating them, because I already have one ESPHome device running, and it was easier to lower my cognitive load to use something I already kinda know. After all, the plan is to replace the Kasa plug once we take out the Christmas tree, which would remove not one but two integrations from Google and Amazon (and one from Home Assistant).

All of these options can be flashed the first time around with tuya-convert, even though the procedure ended up not being totally clear to me — although in part that was caused by my… somewhat difficult set up. This was actually part of the requirements. Most smart home devices appear to be built around ESP8622 or ESP32, which means you can, with enough convincing, flash them with ESPHome (I’m still convincing my acrylic lamp circuit board), but quite a few require you have physical access to the serial port. I wanted something I could flash without having to take it apart.

The way this works, in very rough terms, is that the Tuya-based devices can be tricked into connecting to a local open network, and from there with the right protocol, they can be convinced to dump their firmware and to update it with an arbitrary firmware binary. The tuya-convert repository includes pretty much all the things you need to set this up, neatly packaged in a nearly user-friendly way. I say nearly, because as it turns out there’s so much that can go wrong with it, that the frustration is real.

The Process.

Part 1: WiFi.

First of all, you need a machine that has a WiFi adapter that supports Access Point mode (AP mode/hostapd mode, those are the keywords to look for it). This is very annoying to know for sure, because manufacturers tend to use the same model number across hardware revisions (that may entirely change the chipset) and countries — after all, Matthew ended up turning to Xbox One controller adapters! (And as it turns out, he says they should actually support that mode, with a limited range.)

The usual “go-to” for this is to use a laptop which also has an Ethernet port. Unfortunately I don’t have one, and in particular, I don’t have a Linux laptop anymore. I tried using a couple of Live Distros to set this up, but for one reason or another they were a complete bust (I couldn’t even type on Ubuntu’s Terminal, don’t even ask.)

In my case, I have a NUC on top of my desk, and that’s my usual Linux machine (the one I’m typing on right now!) so I could have used that… except I did disable the WiFi in the firmware when I got it, since I wired it with a cable, and I didn’t feel the need to keep it enabled. This appears to have been a blessing in disguise for a while, or I would have been frustrated at one of the openSUSE updates in December, when a kernel bug caused the system to become unusable with the WiFi on. Which is what happened when I turned the WiFi on in the firmware.

Since I don’t like waiting, and I thought it would generally be a good idea to have at least one spare USB WiFi dongle at home (it would have turned useful at least once before), I went and ordered one on Amazon that people suggested might work. Except it probably got a hardware revision in the middle, and the one I received wasn’t suitable — or, some of the reports say that it depends on the firmware loaded on it; I really didn’t care to debug that too much once I got to that stage.

Fast forward a few weeks, the kernel bug is fixed, so I tried again. The tuya-convert script uses Docker to set up everything, so it sounded like just installing Docker and docker-compose on my openSUSE installation would be enough, right? Well, no. Somehow the interaction of Docker and KVM virtual machines had side effects on the networking, and when I tried I both lost connectivity to Home Assistant (at least over IPv6), and the tuya-convert script kept terminating by itself without providing any useful information.

So instead, I decided to make my life easier and more difficult at the same time.

Part 1.1: WiFi In A Virtual Machine

I didn’t want Docker to make a mess of my networking setup. I also wasn’t quite sure of what tuya-convert would be doing on my machine (yes, it’s open source, but hey I don’t have time to audit all of it). So instead of trying to keep running this within my normal openSUSE install, I decided to run this in a virtual machine.

Of course I need WiFi in the VM, and as I said earlier, I couldn’t just pass through the USB dongle, because it wouldn’t work with hostapd. But modern computers support PCI pass-through, when IOMMU is enabled. My NUC’s WiFi supports hostapd, and it’s sitting unused, since I connect to the network over a cable.

The annoying part was that for performance issues, IOMMU is disabled by default, at least for Intel CPUs, so you have to add intel_iommu=on for the option of passing through PCI devices to KVM virtual machines to be available. This thread has more information, but you probably don’t need all of it, as that focuses on passing through graphic cards, which is a much more complicated topic.

The next problem was what operating system to run in the VM itself. At first I tried using the LiveDVD of openSUSE — but that didn’t work out: the Docker setup that tuya-convert uses is pretty much installing a good chunk of Ubuntu, and it runs out of memory quickly, and when it does it throws a lot of I/O errors from the btrfs loaded into memory. Oops.

Missing a VM image of openSUSE Tumbleweed, I decided to try their image for JeOS instead, which is a stripped down version meant to be used in virtualized environments. This seemed to fit like a glove at first, but then my simplicity plans got foiled by not realizing that usually virtualized environments don’t care for WiFi. Although the utter lack of NetworkManager or any other WiFi tooling turned out to be handy to make sure that nothing tried to steal the WiFi away from tuya-convert.

In addition to changing the kernel package with a version that cares about WiFi drivers, you need to install the right firmware package for the card. After that, at least the first part is nearly taken care of — you will most likely need a few more tools, such as Git and your editor of choice. And of course, Docker and docker-compose.

And then, do yourself a favour, and turn off firewalld entirely in your virtual machine. Maybe I should have said earlier “Don’t let the VM be published to the Internet at large”, but I hope if you get to try to pass-through a WiFi device, you knew better than doing that anyway. The firewall is something that is not obvious is going to get in your way later when you set to run tuya-convert, and it’ll make the whole setup fail silently in the hardest way possible to debug.

Indeed, when I looked for my issues I found tons of posts, issues, blogs all complaining about the same symptom I had, which was all caused by having a firewall in place. The tuya-convert script does a lot of things to set up stuff, but it can’t easily take down a firewall, and that is a biggie.

Indeed, and I’ll repeat that later, at some point the instructions tell you to connect some other device to their network and suggest otherwise it might not be working. My impression is that this is done because if it doesn’t work, you shouldn’t try taking the next steps yet. But the problem is that there is no note anywhere to help you if it doesn’t work — and the reason for it failing is likely the firewall stopping the DHCP server from receiving the requests. Oops.

Part 2: The Firmware Blob

ESPHome configurations are… sometimes very personal. I have found one for the SP111 on the Home Assistant forums and adapted it, but… I don’t really feel like recommending that one. So I’m afraid I won’t take responsibility for how you configure your ESPHome firmware for the plug.

Also, once you have ESPHome on the device, changing the config is nearly trivial, from the Home Assistant integration, so I feel it’s important to have something working at first, and then worry about perfecting it.

I think someone will be confused here on why am I jumping on configuring the firmware blob before we got to convert the device to use it. The reason for that is that you want to have the binary file (either built locally or generated with the Home Assistant integration and downloaded), and you put it into tuya-convert/files/, you will be able to directly flash that version, without going through the intermediate step of using one of the bundled firmware just to be able to update to an arbitrary firmware. But to do that, it needs to happen before you complete the Docker setup.

So, find yourself a working config for the device on the forums (and if you find one that is maintained and templated, so that one can just drop it in and just configure the parameters, please let me know), and generate your ESPHome firmware from there.

Also note that the firmware itself identifies the specific device. This means you cannot flash more than one device with the same firmware or you’ll have quite the headache to sort them out afterwards. Not saying it isn’t possible, but I just found it easier to make the firmware for the devices I was going to flash, and then load each one. As usual, my favourite tool to remember what is what would be my label maker, so that I don’t mix up which one I flashed with which binary.

Part 3: The Conversion

Okay so here’s the inventory of what you should have by this point before we move on to the actual conversion:

  • a virtual machine with a passed-through WiFi card that is supported by hostapd;
  • an operating system in the VM with the drivers for the WiFi, Docker, docker-compose, and no firewall;
  • a checkout of tuya-convert repository;
  • one or more ESPHome firmware binary files (one per device you want to flash), in the files/ directory of the checkout.

Only at this point you can go and follow the instruction of tuya-convert: create Docker image, setup docker-compose, and run the image. The firmware files need to be added before creating the docker image, because docker-compose does not bind the external files/ directory at all.

Once the software starts, it’ll ask you to connect another device that is not the plug to the WiFi. I’m not entirely sure if it’s just for diagnostics, but in either case, you should be able to connect to the network — the password should be flashmeifyoucan, although I don’t think I’ve seen that documented anywhere except when googling around for other people having had issues with their process.

If you try this from your phone, you should be prompted to login into the WiFi network through a captive portal — the portal is just a single page telling you that the setup is completed. If your phone gets stuck in the “Obtaining IP Address” phase, like mine did, make sure you really took down the firewall. This got me stuck for a while because I thought that the Docker itself controlled the whole firewall settings — but that does not appear to be the case.

Final Thoughts

I guess that this guide is not going to be very useful, with the new versions of the SP111 not being supported by tuya-convert (and not clear if it can be supported), but since I have two plugs still unused, it helps me to have written down the process to avoid getting myself stuck again.

The plugs appear to have configurable sensors for voltage, amperage, and total wattage used — and the configuration of those is why I’m not comfortable sharing the config I’m using: I took someone’s from a forum post but I don’t quite agree with some of the choice made, some of the values appear fairly pointless to me.

Voltage monitoring would have been an useful piece of information when I was back in Italy — those who read the blog a long time ago might remember that the power company over there didn’t really have any decent power available. Over here it feels like it’s very stable, so I doubt we’ll notice anything useful with these.

Having Smart Home devices that don’t rely on cloud services is much more comfortable than otherwise. I do like the idea of being able to just ask one of the voice assistants to turn off the subwoofer while I’m playing Fallout 76, for sure — but it’s one thing to have the convenience, and another to depend on it to control it. And as I said some time ago, I disagree with the assertion that there cannot be a secure and safe IoT Smart Home (and yes, “secure” and “safe” are two separate concepts).

As for smart plugs in particular? I’m still not entirely sold, but I can see that there definitely are devices where trying to bring the smart in the device is unlikely to help. Not as many though — it’s still a problem to find something that cannot be served better by more fine-grained control. In the case of the subwoofer, most of the controls (volume, cross-over, phase) are manual knobx on the back of the device. Would it have made sense to have a “smart subwoofer” that can tweak all of those values from the Home Assistant interface? I would argue yes — but at the same time, I can see in this case an expense of £10 for a smart plug beats the idea of replacing the subwoofer entirely.

I honestly have doubts about the Christmas tree lights as well. Not that I expect to be able to control them with an app, but the “controller” for them seems to be fairly standard, so I do expect if I search AliExpress for some “smart” controller for those I will probably find something — the question is whether I would find something I can use locally without depending on an external cloud service from an unknown Chinese brand. So maybe I’ll go back to one of my oldest attempts at electronics (13 years ago!) and see what I can find.

By the way, if you’re curious what else I am currently planning to use these smart plugs on… I’m playing with the idea of changing my Birch Books to use 12V LEDs – originally meant for Gunpla and similar models – and I was thinking that instead of leaving it always-on, I can just connect it with the rest of the routines that we use to turn the “living” items on and off.

Home Assistant and CGG1 Sensors

You may or may not know that I’m one of the few tech people who don’t seem to scream against IoT and I actually have quite a few “smart home” devices in my apartment — funnily enough, one less now that we don’t have normal heating at home and we had to give up the Nest Thermostat. One of the things that I have not set up until now was Home Assistant, simply because I didn’t really need anything from it. But when we moved to this flat, I felt the need to have some monitoring of humidity and temperature in the “winter garden” (a part of the flat that is next to the outside windows, and is not heated), since we moved up in floors and we no longer have the “repair” of being next to another building.

After talking with my friend Srdjan, who had experience with this, I settled on ordering Xiaomi-compatible “ClearGrass” CGG1 sensors from AliExpress. These are BLE sensors, which mean they can broadcast their readings without requiring active connections, and they have a nice eInk display, which wastes very little power to keep running. There’s other similar sensors, some cheaper in the short run, but these sounded like the right compromise: more expensive to buy, but cheaper to run (the batteries are supposed to last six months, and cost less than 50p).

Unfortunately, getting them to work turned out to be a bit more complicated than either of us planned at the beginning. The sensors arrived on a day off, less than two weeks after ordering (AliExpress is actually fairly reliable when ordering to London), and I spent pretty much the whole day, together with Srdjan over on videocall, to get them to work. To save this kind of pain for the next person who come across these issues, I decided to write this up.

Hardware Setup Advices

Before we dig into the actual configuration and setup, let me point out a few things about setting up these devices. The most annoying part is that the batteries are, obviously, isolated to avoid running out in shipping, but the “pull out tab” doesn’t actually pull out. It took quite a bit of force to turn the battery compartment door, open it up, and then re-set it in. Be bold in taking them out.

The other advice is not from me but from Srdjan: write the MAC address (or however it’s called in Bluetooth land) on each of the sensors. Because if you only have one sensor, it’s very easy to tell which one it is, but if you bought more (say, four like I did), then you may have issues later to identify which one is which. So it’s easier to do while you’re setting them up, by turning them on one at a time.

To tell what’s the address, you can either use an app like Beacon Simulator, and listen to the broadcast, or you can wait until you get to a point when you’re ready to listen to the broadcast, later in the process. I would recommend the former, particularly if you live in a block of flats. Not only there’s a lot of stuff that broadcast BLE beacons, but nowadays pretty much every phone is broadcasting them as well due to the Covid 19 Exposure Notifications.

And to make sure that you don’t mix them up, I definitely suggest to use a label maker — and if you are interested in the topic, make sure to check out the Museum of Curiosity Series 13, Episode 6.

Finally, there’s a bit of a spoiler of where this whole process is going to end up going to — I’m going to explicitly suggest you avoid using USB Bluetooth dongles, and instead get yourself an ESP32 kit of some kind. ESP32 devkits are less than £10 on Amazon at the time of writing, and you can find them even cheaper on AliExpress — and they will be much more useful, as I’ll go ahead and tell you.

Home Assistant Direct Integration

So first of all, we spent a lot of time mucking around with the actual Home Assistant integrations. There’s a mitemp_bt sensor integration in Home Assistant, but it’s an “active” Bluetooth implementation, that is (supposedly) more power hungry, and require associating the sensors with the host running Home Assistant. There’s also an alternative implementation, that is supposed to use passive BLE scans.

Unfortunately, even trying to install the alternative implementation turned out to be annoying and difficult — the official instructions appears to expect you install another “store-like” interface on top of Home Assistant, which appears to not be that easy to do when you use their “virtual appliance” image in the first place. I ended up hacking it up a bit, but got absolutely nothing out of it: there isn’t enough logging to know what’s going on at any time, and I couldn’t tell if any packet was even received and parsed.

There is also a clear divergence between the Home Assistant guidelines on how to build new integration, and the way the alternative implementation is written — one of the guides (which I can’t now find easily, and that might speak to the reason for this divergence) explicitly suggests not to write complex parsing logic in the integration, and instead build an external Python library to implement protocols and parsers. This is particularly useful when you want to test something outside of Home Assistant, to confirm it works first.

In this case, having a library (and maybe a command line tool for testing) would have made it easier to figure out if the problem with the sensors was that nothing was received, or that something was wrong with the received data.

This was made more annoying too by the fact that for this to work, you need a working Bluetooth adapter connected to your Home Assistant host — which in my case is a virtual machine. And the alternative implementation tells you that it might interfere with other Bluetooth integrations, so you’re suggested to keep multiple Bluetooth interfaces, one for each of the integrations.

Now this shouldn’t be too hard, but it is: the cheapest Bluetooth dongles I found on Amazon are based on Realtek chipsets, which while supported by (recent enough) Linux kernels, need firmware files. Indeed the one dongle I got requires Linux 5.8 or later, or it requests the wrong firmware file altogether. And there’s no way to install firmware files in the Home Assistant virtual appliance. I tried, quite a few times by now.

ESPHome Saves The Day

ESPHome is a project implementing firmware (or firmware building blocks, rather) for ESP8266 and ESP32 boards and devices, that integrates fairly easily with Home Assistant. And since ESP32 supports BLE, ESPHome supports Xiaomi-compatible BLE sensors, such as the CGG1. So the suggestion from Srdjan, which is what he’s been doing himself, is to basically use an ESP32 board as a BLE-to-WiFi bridge.

This was easy because I had a bunch of ESP32 boards in a box from my previous experiments with acrylic lamps, but as I said they are also the same price, if not cheaper, than Bluetooth dongles. The one I’m using is smaller than most breadboard-compatible ESP32 boards and nearly square — it was a cheap option at the time, but I can’t seem to find one available to build now. It’s working out well by size, because it also doesn’t have any pin headers soldered, so I’m just going to double-side-tape it to something and give it a USB power cable.

But it couldn’t be as easy as to follow the documentation, unfortunately. While configuring the ESPhome is easy, and I did manage to get some readings almost right away, I found that after a couple of minutes, it would stop seeing any signal whatsoever from any of the sensors.

Digging around, I found that this was not uncommon. There’s two ESPhome issues from 2019: #317 and #735 that report this kind of problems, with no good final answer on how to solve them, and unfortunately locked to collaborators, so I can’t leave breadcrumbs for the next person in there — and it’s why I am now writing this, hopefully it’ll save headaches for others.

The problem, as detailed in #735, is that the BLE scan parameters need to be adjusted to avoid missing the sensors’ broadcasts. I tried a few combinations, and at the end found that disabling the “active” scan worked — that is, letting the ESP32 passively listen to the broadcasts, without trying to actively scan the Bluetooth channels seemed to let it stay stable, now for over 24 hours. And it should also be, as far as I can tell, less battery-draining.

The final configuration looks something like this:

esp32_ble_tracker:
  scan_parameters:
    duration: 300s
    window: 48ms
    interval: 64ms
    active: False

sensor:
  - platform: xiaomi_cgg1
    mac_address: "XX:XX:XX:XX:XX:XX"
    temperature:
      name: "Far Corner Temperature"
    humidity:
      name: "Far Corner Humidity"
    battery_level:
      name: "Far Corner Battery Level"

The actual scan parameters should be, as far as I can tell, ignored when disabling the active scan. But since it works, I don’t dare to touch it yet. The code in ESPhome doesn’t make it very clear if changing those parameters when disabling active scan is entirely ignored, and I have not spent enough time going through the deep stack to figure this out for certain.

The only unfortunate option of having it set up this way, is that by default, Home Assistant will report all of the sensors in the same “room”, despite them being spread all over the apartment (okay, not all over the apartment in this case but in the winter garden).

I solved that by just disabling the default Lovelace dashboard and customizing it. Because turns out it’s much nicer to customize those dashboards than using the default, and it actually makes a lot more sense to look at it that way.

Looking To The Future

So now I have, for the first time, a good reason to use Home Assistant and to play around with it a bit. I actually have interesting “connected home” ideas to go with it — but they mostly rely on getting the pantograph windows working, as they don’t seem currently to be opening at all.

If I’m correct in my understanding, we’ll need to get the building managers to come and fix the electronic controls, in which case I’ll ask for something I can control somehow. And then it should be possible to have Home Assistant open the windows in the morning, assuming it’s not raining and the temperature is such that a bit of fresh air would be welcome (which is most of the summer, here).

It also gives me a bit more of an incentive to finish my acrylic lamps work — it would be easy, I think, to use one of those as a BLE bridge that just happens to have a NeoPixel output channel. And if I’m going “all in” to wire stuff into Home Assistant, it would also allow me to use the acrylic lamps as a way to signal for stuff.

So possibly expect a bit more noise on this front from me, either here or on Twitter.

Update 2020-12-06: The sensors I wrote about above are an older firmware CGG1 sensors that don’t use encryption. There appear to be a newer version out that does use encryption, and thus require extracting a bind key. Please follow the reported issue for updates.

Update 2021-06-26: I forgot to update this, but ESPHome 1.17.1 and later support encrypted CGG1 out of the box as my pull request was merged.