Remote Debugging With GDB; part 3: SWD

As part of my Gone With The Wind side project I found myself using something that I hadn’t used in many years: GDB with remote debugging, which is something I have done many times in the past, both on servers and on embedded system, and last described in 2010.

In this case though, I’m going to be attaching to an MCU as part of a development kit. Which particular development kit only counts up to a point: as I wrote in the introduction to my USB-PD adventure, I started looking at UPD301C (despite it being “Not Recommended for new designs”) since I could use it to implement Birch Books in just two chip, which ended up with me buying an EV11L78A devkit; but I also ended up buying an SAMD20 Xplayer PRO, and I had many other embedded boards from my dabbling with Circuit Python a few years back. And what I’m about to write applies almost for all of these.

Indeed, the reason why I felt comfortable with ordering the UPD301C in the first place was that I knew I could jump into it right away thanks to the SEGGER J-Link EDU Mini I bought back then to un-brick an Adafruit Metro. So indeed, my first set of remote debugging happened with this device — but I decided, eventually, not to continue using it because it didn’t feel right. While definitely my project is not for profit, the fact that my contributions to Zephyr will carry over Meta’s copyright made me feel uncomfortable with the situation.

Let’s Talk (To The) Hardware

When debugging firmware running on a MCU, you generally need some way to access the MCU itself — this is usually done through a JTAG or SWD (Single-Wire Debug) connector, though some of the devkits provide an inline access to it. What you need to be able to use these ports is a hardware interface that can talk the underlying debugging protocol and expose it somehow.

SEGGER’s J-Link series, which I linked above, are one such hardware interface. The EDU Mini I linked to is a simplified version of their higher-end one, but it still allows the remote debugging I’m talking about here. It effectively plugs into a USB port on one side, and on the devkit’s debug connector on the other. There’s two main options to use this interface for debugging: the proprietary software package from SEGGER (which integrates with other software such as Microchip’s IDE) or openocd. Either of these can then act as middleware for GDB to access the device.

I personally found SEGGER’s software a bit clunky, but serviceable — it’s par of the course of my experience doing firmware development back in Italy in the early noughties, so I can’t really fault SEGGER for it. And eventually once I got my way through most of its idiosyncrasies, I had a fairly reliable path to flash and step through my code.

Most of their software has both UI and CLI versions, and in particular the GDB server implementation generates a copy-pastable command from the UI configuration, to run it as part of a script or in a command-line session, after confirming the configuration. I also liked the option of separating the system connected to the interface and the one accessing it by letting it be shared over the network, although I haven’t tried if it supported IPv6.

So for example for the target I’m working on, starting the JLink GDB server can be done with

JLinkGDBServer \
  -select USB \
  -device ATSAMD20E16A \
  -endian little
  -if SWD -speed 4000 \
  -noir \
  -noLocalhostOnly \
  -nologtofile \
  -port 2331 \
  -SWOPort 2332 \
  -TelnetPort 2333

And then you can connect to it via gdb with

target extended-remote :2331Code language: CSS (css)

As I said, the EDU Mini version is explicitly targeted to non-professional users. Their spelling of the acceptable usage for the device is quite clear, and honestly speaking, fair. Unlike the Jetpack phrasing of “commercial sites”, the main restriction is about profits — but that still made me a bit uncomfortable as I said. I very briefly considered upgrading to their full edition, but even the J-Link BASE Classic is close to €400! And that is without any adapters.

But then I remembered about Black Magic Probe which is basically designed for exactly what I need. This is a hardware interface capable of interfacing directly with GDB from the get-go, with target discovery integrated in the GDB cli, and completely open source. And it’s a lot cheaper than the J-Link (though, to be clear, there’s a lot more meat in the J-Link hardware — I just don’t need any of that.)

This means you only need to configure things from within GDB:

target extended-remote /dev/ttyACM0
monitor swdp_scan
attach 1

In addition to the SWD interface and the GDB server, the Probe comes with a 4-wire serial port (RX/TX, GND and VCC) which could, for instance, be used to unbrick a TS-100 module, just as a random example. (What? Me? Bricking my TS-100? Nah — I’m sure I had a totally valid reason to upgrade to a Pinecil!) And until you have had to connect two separate cables to reach into the same board you don’t know how valuable that feature is (it was literally the feature that got me to order the Probe whether I technically needed it or not!)

Finally, some devkits, such as the SAMD 20 Xplained PRO (which I bought to test my first Zephyr attempts) come with an embedded debugger, so require no additional interface. This is definitely more common for devkits, rather than final products, since it’s literally building a second MCU and firmware into each board.

In the case of the Atmel EDBG, it appears the easiest path is to use openOCD as a middleware to the target, which means starting it with a command similar to

OPENOCD_BASE=/usr/share/openocd/scripts
BOARD=atmel_samd20_xplained_pro
openocd -f ${OPENOCD_BASE}/board/${BOARD}.cfgCode language: JavaScript (javascript)

And then connecting to it from GDB with

target extended-remote :3333Code language: CSS (css)

In-between me starting to writing this post, and the post actually being scheduled for publishing, Raspberry Pi announced their own Debug Probe based around the RP2040. This provides a (much) smaller set of features of the Black Magic Probe, to my understanding, since it’s only able to speak SWD rather than JTAG. On the other hand, SWD is all you need for most hobbyists ARM boards, and the probe is only £12, and supports CMSIS-DAP (the same interface used by the EDBG above.)

If you’re not boycotting Raspberry Pi, it might be a good option to get started with embedded firmware development.

Yeah, Okay, But What’s Where?

This is all good, but why on Earth was I even needing this much debugging, rather than my pretty much usual approach of flashing a firmware and put some serial output on it? I mean, that works on 8051s too! Well, the answer is that I was trying to figure out what I was doing wrong, that my devkit kept running at a 1/48th of the speed it was meant to (answer: I should have changed the source of main clock from the board config I copied when bringing up the EV11L78A.)

But before identifying that problem, I wanted to exclude I misconfigured some register — except that after a while I got tired of remembering and calculating register addresses by hand, and expected that there had to be better way. The answer was “yes, there is a better way” but at the same time it’s not quite as trivial, at least where Zephyr is involved.

When I asked on Mastodon, Scott had the answer: SVD files (part of the CMSIS – Common Microcontroller Software Interface Standard – specifications) include information about all of the registers available on a compatible MCU, with semantics for each bit or bitstruct. And there is a tool called PyCortexMDebug that can parse those descriptor files and provide you easier lookup of the peripheral’s registers.

But this is where I hit a speedbump, which relates to the selection of the GDB binary. Technically speaking, you can use any GDB binary that is compatible with the architecture you need to debug (namely in my case, almost exclusively ARM.) But practically speaking if you’re working with Zephyr like I’m doing, you will want to use the binaries that come as part of the Zephyr SDK. Indeed, Zephyr provides not one, but two GDB binary for each architecture: one with Python built-in, and one without. This makes sense, since they want to cover as many use cases as possible, and it’s not unlikely that some of the people needing the SDK wouldn’t have a compatible Python version at hand.

At the time I’m writing this, Zephyr SDK comes with GDB liked with Python 3.8 — which thankfully works on openSUSE Tumbleweed, which is the distro I’m doing most of my work on at the moment. To a point.

When I tried importing PyCortexMDebug from a arm-zephyr-eabi-gdb-py session, it failed. The reason was that the tool required lxml which I hadn’t installed for Python 3.8 — it sounds like an easy fix but it wasn’t: while openSUSE still provides Python 3.8, they do not provide pip for it, which means I couldn’t just install lxml. So my route was to replace the lxml usage with ElementTree (which is part of the Standard Library) so that the Zephyr SDK GDB would be able to import it — and it worked! Until openSUSE also removed setuptools and it failed import again, which I had to work around a bit.

So now I have a pending pull request to make the tool work with just standard library components. It works and it makes sense to a point, but there’s a catch: this is basically duplicating (badly) the CMSIS parser for SVD that already exist — but trying to use it would require straying from the Standard Library approach, obviously.

This is one of the problems I haven’t really quite solved myself, mostly because there’s two different ways to see this:

• Expect the Python version used in GDB to be full-featured: this would mean that the problem is trying to use Zephyr SDK (which links to Python 3.8) on openSUSE (which does not support Python 3.8 as a full-featured version). If this is the case, it would require either changing the host distro (CentOS 8 would work I think) or use a different GDB built against a newer Python version.
• Expect the Python version used in GDB to be basic and only assume presence of the Standard Library (as is the case for the Zephyr SDK on openSUSE) and consider it just a scripting helper, closer to LUA than a full-fledged application framework. In which case the code loaded in GDB has to be reduced to a minimum.

These different views lead to quite different options and development. My current take is that it would make sense to treat the embedded Python support is just a scripting engine, and thus one of the things I’ve considered, if needed, is to separate the parsing of SVD files from the in-GDB code. Indeed, it’s only a fraction of the SVD file that is usually needed for providing peripheral register mnemonics, and even wanting to provide more functionality than that, I would personally have rather expected a two-step process, where you feed the SVD file to a generator, that produces the code that is loaded by GDB.

Leaving aside these particular problems, once you get the right SVD file (in my case I needed the ATSAMD20E14 one that can be found on the Microchip Technology Packs Repository) you can load this in the GDB session, and then you get to access this from the command line:

(gdb) source PyCortexMDebug/scripts/gdb.py
(gdb) svd_load ~/upd301c/support/ATSAMD20E16.svd
Loading SVD file ATSAMD20E16.svd...
(gdb) set mem inaccessible-by-default off  # for Black Magic Probe
(gdb) svd SERCOM1 USART_INT BAUD
Fields in SERCOM1 > USART_INT > BAUD:
        BAUD:  63019  Baud ValueCode language: JavaScript (javascript)

Having the mnemonic name available makes it so much handier to inspect the state of registers and make sure they are configured the way you expect them to. Do note that, unlike the SEGGER version, you do need one extra command for the Black Magic Probe to allow you to access non-RAM memory.

And with a bit more scripting from there, you can also have it report a “diff” in state between initializations, for instance when you want to compare how the original firmware of a certain devkit behaves compared with the one you’re building yourself…

Putting It All Together

All of this is a bit of a mouthful — how do you make sure it all fits together? There’s a lot of moving parts, there’s variable device names, and there’s a lot of steps to be taken. The good thing is that you can actually write this in a GDB script file, and have it execute all the default commands for you.

And as documented in the FAQ, you can use Linux’s persistent, predictable device names to avoid fiddling with the correct device name. Which is how I ended up with the following script:

target extended-remote /dev/serial/by-id/usb-Black_Magic_Debug_Black_Magic_Probe_v1.8.2_XXXXX-if00
monitor swdp_scan
attach 1
source PyCortexMDebug/scripts/gdb.py
svd_load ATSAMD20E16.svd
file zephyrproject-rtos/zephyr/build/zephyr/zephyr.elf

And I have an alias zephyr-gdb (well actually ev11l78a-gdb) that runs the correct Zephyr SDK GDB binary running the script with -x ev11l78a-bmp.

Both Probe and J-Link allow to flash the firmware onto the device by using the load command after selecting the file, which is a great way to just have everything in one place. And since you can technically run the west build system from the GDB shell, there’s no reason to have much else than two tmux windows with GDB and the serial output for me to access the MCU (well, and the VSCode editor.)

I think things feel a lot easier nowadays than what I used to work with in my home office back in Italy, particularly given how open source software and hardware are making it a lot more affordable for hobbyists to have access to features that used to be almost exclusively available to high end professionals. I hope that, small as they are, my contributions will also enable others to build something that they couldn’t build as easily before.