Software Defined Remote Control

A number of months ago I spoke about trying to control a number of TV features in Python. While I did manage to get some of the adapter boards that I thought I would use printed, I hadn’t had the time to go and work on the software to control this before we started looking for a new place, which meant I shelved the project until we could get to the new place, and once we got there it was a matter of getting settled down, and then, … you got the idea.

As it turns out, I had one week free at the end of November — my employer decided to give three extra days on the (US) Thanksgiving week, and since my birthday was at the end of the week, I decided to take the remaining two days off myself to make it a nice nine days contiguous off. Perfect timeframe to go and hack on some projects such as that.

Also, one thing changed significantly since the time I started thinking about this: I started using Home Assistant. And while it started mostly as a way for me to keep an eye on the temperature of the winter garden, I found that with a bit of configuration, and a pull request, changing the input on my receiver with it was actually easier than using the remote control and trying to remember which input was mapped to what.

That gave me finally the idea of how to implement my TV input switch tree: expose it as one or more media players in Home Assistant!

Bad (Hardware) Choices

Unfortunately, as soon as I went to start implementing the switching code, I found out that I had made a big mistake in my assumptions: the Adafruit FT232H breakout board does not support PWM outputs, including the general time-based pulsing (without a carrier frequency). Indeed, while the Blinka library can technically support some of the features, it seems like none of the Linux-running platforms would be able to manage that. So there goes my option of just using a computer to drive the “fake remote” outputs directly. Well, at least without rewriting it in some other language and find a different way to send that kind of signals.

I looked around for a few more options, but all of it ended up being some compromise: MicroPython doesn’t have a very usable PulseOut library as far as I could tell; Arduino-based libraries don’t seem to allow two outputs to happen at roughly the same time; and as I’m sure I already noted in passing, CircuitPython lacks a good “secondary channel” to be instructed from a computer (the serial interface is shared with the REPL control, and the HID is gadget-to-host only).

After poking around a few options and very briefly considering writing my own C version on an ATmega, I decided to just go for the path of least resistance, and go back to CircuitPython, and try to work with the serial interface and its “standard input” to the software.

The problem with doing that is that the Ctrl-C command is intended to interrupt the command, and that means you cannot send the byte 0x03 un-escaped. At the end I thought about it, and decided that CircuitPython is powerful enough that just sending the commands in ASCII wouldn’t be an issue. So I decided to write a simplistic Flask app that would take a request over HTTP and send the command via the serial port. It worked, sort of. Sometimes while debugging I would end up locking the device (a Trinket M0) in the REPL, and that meant the commands wouldn’t be sent.

The solution I came up with was to reset the board every time I started the app, by sending Ctrl-C and Ctrl-D (0x03, 0x04) to force the board to reset. It worked much better.

Not-Quite-Remote Controlled HDMI Switch

After that worked, the problem was ensuring that the commands sent actually worked. The first component I needed to send the commands to was the HDMI switch. It’s a no-brand AliExpress-special HDMI switch. It has one very nice feature for what I need to do right now. It obviously has an infrared remote control – one of those thin, plasticky domes one – but it particularly has the receiver for it on a cord, which is connected with a pretty much standard 3.5mm “audio jack”.

This is not uncommon. Randomly searching Amazon or AliExpress for “HDMI switch remote” can find you a number of different, newer switches that use the same remote receiver, or something very similar to it. I’m not sure if the receivers are compatible between each other, but the whole idea is the same: by using a separate receiver, you can stick the HDMI switch behind a TV, for instance, and just make the receiver poke from below. And most receivers appear to be just a dome-encased TSOP17xx receiver, which is a 3-pin IC, which works great for a TRS.

When trying this out, I found that what I could do would be to use a Y-cable to allow both the original receiver and my board to send signals to the switch — at which point, I can send in my own pulses, without even bothering with the carrier frequency (refer to the previous post for details on this, it’s long). The way the signal is sent, the pulses need to ground the “signal” line (that is usually at 5V); to avoid messing up the different supplies, I paired it on an opto-coupler, since they are shockingly cheap when buying them in bulk.

But now that I tried setting this up with an input selection, I found myself not able to get the switch to see my signal. This turned out to require an annoying physical debugging session with the Saleae and my TRRS-to-Saleae adapter (that I have still not released, sorry folks!), which showed I was a bit off on the timing of the NEC protocol the switch used for the remote control. This is now fixed in the pysirc library that generates the pulses.

Once I got the input selector working for the switch with the Flask app, I turned to Home Assistant and added a custom component that exposes the switch as a “media_player” platform. In a constant state of “Idle” (since it doesn’t have a concept of on or off), it allowed me and my wife to change the input while seeing the names of the devices, without hunting for the tiny remote, and without having to dance around to be seen by the receiver. It was already a huge improvement.

But it wasn’t quite enough where I wanted it to be. In particular, when our family calls on Messenger, we would like to be able to just turn on the TV selected to the right input. While this was partially possible (Google Assistant can turn on a TV with a Chromecast), and we could have tried wiring up the Nabu Casa integration to select the input of the HDMI switch, it would have not worked right if the last thing we used the TV for was the Nintendo Switch (not to be confused with the HDMI switch) or for Kodi — those are connected via a Yamaha receiver, on a different input of the TV set!

Enter Sony

But again, this was supposed to be working — the adapter board included a connection for an infrared LED, and that should have worked to send out the Sony SIRC commands. Well, except it didn’t, and that turned out to be another wild goose chase.

First, I was afraid that when I fixed the NEC timing I broke the SIRC ones — but no. To confirm this, and to make the rest of my integration easier, I took the Feather M4 to which I hard-soldered a Sony-compatible IR LED, and wrote what is the eponymous software defined remote control: a CircuitPython program that includes a few useful commands, and abstractions, to control a Sony device. For… reasons, I have added VCR as the only option beside TV; if you happen to have a Bluray player by Sony, and you want to figure out which device ID it uses, please feel free.

It might sound silly, but I remember seeing a research paper in UX from the ’90s of using gesture recognitions on a touchpad on a remote control to allow more compact remote controls. Well, if you wanted, you could easily make this CircuitPython example into a touchscreen remote control for any Sony device, as long as you can find all the right device IDs, and hard code a bunch of additional commands.

So, once I knew that at least on the software side I was perfectly capable of control the Sony TV, I had to go and do more hardware debugging, with the Saleae, but this time with the probes directly on the breadboard, as I had no TRS cable to connect to. And that was… a lot of work, to rewire stuff and try.

The first problem was that the carrier frequency was totally off. The SIRC protocol specifies a 40kHz carrier frequency, which is supposedly easier to generate than the 38kHz used by NEC and others, but somehow the Saleae was recording it as a very variable frequency that oscillated between 37kHz and 41kHZ. So I was afraid that trying to run two PWM outputs on the Trinket M0 was a bad idea, even if one of them was set to nought hertz — as I said, the HDMI switch didn’t need a carrier frequency.

I did toy briefly with the idea of generating the 40kHz carrier wave separately, and just gating it to the same type of signal I used for the HDMI switch. Supposedly, 40kHz generators are easy, but at least for the circuits I found at first glance, it requires a part (640kHz resonator) that is nearly impossible to find in 2020. Probably fell out of use. But as it turn out it wouldn’t have helped.

Instead, I took another Feather. Since I ran out of M4, except for the one I hardwired already an IR LED to, I instead pulled up the nRF52840 that I bought and barely played with. This should have been plenty capable to give me a clean 40kHz signal and it indeed was.

At that point I noticed another problem, though: I totally screwed up the adapter board. In my Feather M4, the IR LED was connected directly between 3V and the transistor switching it. A bit out of spec, but not uncommon given that it’s flashed for very brief impulses. On the other hand when I designed the adapter, I connected it to the 5V rail. Oops, that’s not what I was meant to be doing! And I did indeed burn out the IR LED with it. So I had to solder a new one on the cable.

Once I fixed that, I found myself hitting another issue: I could now turn on and off the TV with my app, but the switch stopped responding to commands either from the app or from the original remote! Another round of Saleae (that’s probably one of my favourite tools — yes I splurged when I bought it, but it’s turning out to be an awesome tool to have around, after all), and I found that the signal line was being held low — because the output pin is stuck high…

I have not tried debugging this further yet — I can probably reproduce this without my whole TV setup, so I should do that soonish. It seems like opening both lines for PWM output causes some conflicts, and one or the other end up not actually working. What I solved this with was only allowing one command before restarting the Feather. It meant taking longer to complete the commands, but it allowed me to continue with my life without further pain.

One small note here: since I wasn’t sure how Flask concurrency would interact with accessing a serial port, I decided to try something a bit out of the ordinary, and set up the access to the Feather via an Actor using pykka. It basically means leaving one thread to have direct access to the serial port, and queue commands as messages to it. It seems to be working fine.

Wrapping It All Up

Once the app was able to send arbitrary commands to the TV via infrared, as well as changing the input of the HDMI, I extended the Home Assistant integration to include the TV as a “media_player” entity as well. The commands I implemented were Power On and Off (discrete, rather than toggle, which means I can send a “Power On” to the TV when it’s already on and not bother it), and discrete source selection for the three sources we actually use (HDMI switch, Receiver, Commodore 64). There would be a lot more commands I could theoretically send, including volume control, but I can already access those via the receiver, so there’s no good reason to.

After that it was a matter of scripting some more complicated acts: direct selection of Portal, Chromecast, Kodi, and Nintendo Switch (which are the four things we use the most). This was easy at that point: turn on the TV (whether it was on or not), select the right input on either the receiver or the switch, then select the right input ion the TV. The reason why the order seems a bit strange is that it takes a few seconds for the TV to receive commands after turning on, but by doing it this way we can switch between Chromecast and Portal, or Nintendo Switch and Kodi, in pretty much no time.

And after that worked, we decided the $5/month to Nabu Casa were worth it, because that allows us to ask Alexa or Google Assistant to select the input for us, too.

Eventually, this lead me to replace Google’s “Turn off the TV” command in our nightly routine to trigger a Home Assistant script, too. Previously, it would issue the command to the Chromecast, routing through the whole Google cloud services between the device that took the request and the Chromecast. And then the Chromecast would be sending the CEC command to power off… except that it wouldn’t reach the receiver, which would stay on for another two hours until it finally decided it was time to turn off.

With the new setup, Google is triggering the Home Assistant script, and appears to do that asynchronously. Then Home Assistant sends the request to my app, that then sends it to the Feather, that sends the power off to the TV… which is also read by the receiver. I didn’t even need to send the power off command to the receiver itself!

All in all, the setup is satisfying.

What remains to be done is to try exposing a “Media Player” to Google Home, that is not actually any of the three “media_player” entities I have, but is a composite of them. That way, I could actually just expose the different input trees as discrete inputs to Google, and include the whole play, pause, and volume control that is currently missing from the voice controls. But that can wait.

Instead, I should probably get going at designing a new board to replace the breadboard mess I’m using right now. It’s hidden away enough that it’s not in our face (unlike the Birch Books experiments), but I would still like having a more… clean setup. And speaking of that, I really would love if someone already contributed an Adafruit Feather component for EAGLE, providing the space for soldering in the headers, but keeping the design referencing the actual lines as defined in it.

Controlling Your TV via (Circuit)Python

This is yet another of the pre-announced projects, and possibly one of the most procrastinated ones. I own a Sony Bravia TV I bought in Ireland in 2013, and is still working very well for our needs (we don’t watch that much TV). It’s connected to a Yamaha RX-V475 receiver on one input and a cheap HDMI switch on the other input, because there’s too many devices, although we only use three or four of them most of the time: Chromecast, Portal TV, HTPC, and PlayStation 4. They are split equally between the two inputs. So far, so good.

The problem starts with the fact that sometimes if the TV is turned on by the Chromecast or the Portal, the ARC does not initialize properly, and we hear no audio. The solution is worthy of The IT Crowd: tell the TV to use the internal speakers, then tell it to use the external speakers again — turn off and on the ARC itself. It’s annoying and takes a few keypresses.

What I have been wanting for a while is a way to ask Assistant (or Alexa) to “unfuck the TV” — that is to reset the audio channel for us, recording a macro to do that ARC off/on dance. It was for this reason I bought the Feather M4 last year, but I only ended up starting to work on it just this month.

To make this possible, the first thing I needed was to know the commands sent by the remote, and have a way to replicate them back to the TV. I already had some experience with infrared receivers as, a long time ago in a different life, I maintained the LIRC patchset for Linux 2.6, for a while. I even tried writing Qt (3) bindings for LIRC. I wonder if I can find the source code anywhere. But that was not as useful as I thought.

Thankfully, Ken Shirriff wrote good documentation on the protocol, and linked to further details, and even to a full archive of all the Sony command codes. Which made my life much easier, but not completely easy. While Adafruit has an IRRemote library, it does not correctly interpret Sony SIRC commands. I considered adding the support directly into it, but it turned out to be a bit more invasive than expected, so I ended up instead writing a separate package that included both the decoder and the encoder (this was before I started the new job, so releasing it was easy — but now I’m having to wait a little bit to continue on it.)

Now, once I managed to decode the commands, I need to be able to send them. And here’s where things get interesting. What we usually refer to as commands are a bunch of bits. These are encoded, based on the protocol, as a series of pulses, which are modulated on top of a carrier wave with a certain frequency.

Unfortunately, it turns out that talking to a Sony TV is nowhere near as easy as it might sound. Let’s try to figure out what’s going on by providing a bit of vocabulary. An infrared signal as used by most remote controls usually carries a command and, in most cases, an address to specify which device should take the command (since they are usually used in place where there are multiple devices using infrared remote controls). These are encoded in bits according to the rules of the protocol, and then converted to pulses. These pulses are then applied to a carrier wave of a certain frequency, which defines the speed at which the infrared LED is “blinking” on and off. The LED itself has a wavelength which represent the “colour” of the light in the infrared spectrum.

For Sony’s SIRC, the TV expects a 40kHz carrier, and it seems a 950nm wavelength. It seems like you can use 940nm LEDs but they have worse reception, and only works if they are bright enough. The first batch of LEDs I bought, as well as the pre-soldered module from DFRobot, turned out to not be bright enough for my TV to recognize — so I decided to try with desoldering the LED from a replacement remote control from eBay, which worked fine, so I thought I needed 950nm LEDs — but no, it works with good enough 940nm LEDs, just not with the tiny ones I originally bought from Mouser.

So once I had a way to send arbitrary Sony commands to my TV, I started looking for options to trigger the ARC reset — unfortunately this is proving more complicated than I expected: there’s no command that I could send that would provide me with the ARC menu. Instead I can only bring up the Sync menu reliably — but that menu has different entries depending on whether the selected input has working HDMI CEC, which is extremely annoying.

On the other hand, I did find commands that select directly the different inputs directly, instead of showing up the input selection menu and choosing from there. Which gave me a different idea to start with: while I haven’t given up on the macro for fixing the audio, what I can do is to script input selection across the two-level mux.

I started by wondering if I could send the IR command to the HDMI switcher as well, so that I could select between those two easily — that turned out to be another pile of yaks to shave. The switcher uses the NEC protocol, which has a 38kHz carrier wave, but that turned out not to matter as much (the decoder it uses seem to accept 40kHz just as well) — instead I had a hard time to get it to receive the command because it expected a NEC “repeat signal” to seal the command. I guess that’s going to be a blog post in and by itself.

Now, my original plan was to get something running on the Feather, attach an AirLift wing to give it WiFi, and control that… somehow. I also considered re-coding this with ESP32 and ESPHome, despite it not having an obvious way to send SIRC commands while making sense — it doesn’t represent the commands the way the protocol expects, and the most reasonable way I could find was to generate the pulse sequence, and just sending that raw.

But then I thought it over and realised that, at least for the moment, it makes more sense for me to use an USB-to-GPIO bridge, and control this from a full blown computer — the reason for that is that I would want to be able to open a webpage on my phone, or my wife’s, and select the right input altogether. And while there’s a Python module for controlling the receiver, using that from CircuitPython or MicroPython would probably take some work. And even though I could just control the Feather remotely, via MQTT or USB-Serial, it would still likely be more work than driving this all from the same Flask app locally.

Unfortunately, I don’t have code to show you yet. While my new dayjob has much simpler policies than the last, I need to complete the training before I can start releasing new projects. Hopefully next update will come with code and demos.

XBMC part 2

I have posted about me setting up a new box for XBMC and here is a second part to that post, now that I arrived to Dublin and I actually set it up on my living room as part of my system. There are a few things that needs better be described.

The first problem I had was how to set up the infrared receiver for the remote control. I originally intended to use my Galaxy Note as it has an IR blaster for I have no idea what reason; but then I realized I have a better option.

While the NUC does not, unfortunately, support CEC input, my receiver, a Yamaha RX-V475 comes with a programmable remote controller, which – after a very quick check cat-ing the event input device node – appeared sending signals in the right frequency for the built-in IR sensor to pick it up. So the question was to find a way to map the buttons on the remote to action to XBMC.

Important note: a lot of the documentation out there tells you that the nuvoton driver is buggy and requires to play with /sys files and the DSDT tables. This is outdated, just make sure you use kernel version 3.15 or later and it works perfectly fine.

The first obvious option, which I have seen documented basically everywhere, is to use lirc. Now that’s a piece of software that I know a little too well for comfort. Not everybody knows this, both because I went by a different nickname at the time, and because it happened a long time before I joined Gentoo, and definitely before I started keeping a blog. But as things are, back in the days when Linux 2.5 was a thing, I did the first initial port of the lirc driver to a newer kernel, mostly as an external patch to apply on top of the kernel. I even implemented devfs, since while I was doing that I finally moved to Gentoo, and I needed devfs to use it.

I wanted to find an alternative to using lirc for this and other reasons. Among other things, last time I have used it, I was using it on computer that was not dedicated as an HTPC, so this looked like a much easier task with a single user-facing process in the system. After looking around quite a bit I found that you can make the driver output X-compatible key events instead of IR events by loading the right keymap. While there are multiple ways to do this, I ended up using ir-keytable which comes in v4l-utils.

The remote control only had to be set to send codes for a VDR for the brand “Microsoft” — which I assume puts it in a mode compatible with Windows XP Media Center Edition. Funnily enough they actually put a separate section for Apple TV codes. After that, the RC6/MCE table can be used, and that will send proper keypresses fr things like the arrows and the number buttons.

I only had to change a couple of keys, namely Enter and Exit to send KEY_RETURN and KEY_BACKSPACE respectively, so that they map to actions in XBMC. It would probably be simple enough to change the bindings to XBMC directly, but I find it more reliable for it to send a different key altogether. The trick is to edit /etc/rc_keymaps/rc6_mce to change the key that is sent, and then re-run ir-keytable -a /etc/rc_maps.cfg, and the problem is solved (udev rules are in place so that the map is loaded at reboot).

And this is one more problem solved, now I’m actually watching things with XBMC so it seems to be working fine.