Diabetes Software: the importance of documented protocols

You may remember that just last week I was excited to announce that I had more work planned and lined up for my glucometer utilities, one of which was supporting OneTouch Verio IQ which is a slightly older meter that is still sold and in use in many countries, but for which no protocol is released.

In the issue I linked above you can find an interesting problem: LifeScan discontinued their Diabetes Management Software, and removed it from their website. Indeed instead they suggest you get one of their Bluetooth meters and to use that with their software. While in general the idea of upgrading a meter is sane, the fact that they decided to discontinue the old software without providing protocols is at the very least annoying.

This shows the importance of having open source tools that can be kept alive as long as needed, because there will be people out there that still rely on their OneTouch Verio IQ, or even on the OneTouch Ultra Easy, which was served by the same software, and is still being sold in the US. Luckily at least they at least used to publish the Ultra Easy protocol specs and they are still available on the Internet at large if you search for them (and I do have a copy, and I can rephrase that into a protocol specification if I find that’s needed).

On the bright side, the Tidepool project (of which I wrote before) has a driver for the Verio IQ. It’s not a particularly good driver, as I found out (I’ll get to that later), but it’s a starting point. It made me notice that the protocol was almost an in-between of the Ultra Easy and the Verio 2015, which I already reverse engineered before.

Of course I also managed to find a copy of the LifeScan software on a mostly shady website and a copy of the “cable drivers” package from the Middle East and Africa website of LifeScan, which still has the website design from five years ago. This is good because the latter package is the one that installs kernel drivers on Windows, while the former only contains userland software, which I can trust a little more.

Comparing the USB trace I got from the software with the commands implemented in the TidePool driver showed me a few interesting bits of information. The first being the first byte of commands on the Verio devices is not actually fixed, but can be chosen between a few, as the Windows software and the TidePool driver used different bytes (and with this I managed to simplify one corner case in the Verio 2015!). The second is that the TidePool driver does not extract all the information it should! In particular the device allows before/after meal marking, but they discard the byte before getting to it. Of course they don’t seem to expose that data even from the Ultra 2 driver so it may be intentional.

A bit more concerning is that they don’t verify that the command returned a success status, but rather discard the first two bytes every time. Thankfully it’s very easy for me to check that.

On the other hand, reading through the TidePool driver (which I have to assume was developed with access to the LifeScan specifications, under NDA) I could identify two flaws in my own code. The first was not realizing the packet format between the UltraEasy and the Verio 2015 was not subtly different as I thought, but it was almost identical, except the link-control byte in both Verio models is not used, and is kept to 0. The second was that I’m not currently correctly dropping out control solutions from the readings of the Verio 2015! I should find a way to get a hold of the control solution for my models in the pharmacy and make sure I get this tested out.

Oh yeah, and the TidePool driver does not do anything to get or set date and time; thankfully the commands were literally the same as in the Verio 2015, so that part was an actual copy-paste of code. I should probably tidy up a bit, but now I would have a two-tier protocol system: the base packet structure is shared between the UltraEasy, Verio IQ and Verio 2015. Some of the commands are shared between UltraEasy and Verio IQ, more of them are shared between the Verio IQ and the Verio 2015.

You can see now why I’ve been disheartened to hear that the development of drivers, even for open source software, is done through closed protocol specifications that cannot be published (or the drivers thoroughly commented). Since TidePool is not actually using all of the information, there is no way to tell what certain bytes of the responses represent. And unless I get access to all the possible variants of the information, I can’t tell how some bytes that to me look like constant should represent. Indeed since the Verio 2015 does not have meal information, I assumed that the values were 32-bit until I got a report of invalid data on another model which shares the same protocol and driver. This is why I am tempted to build “virtual” fakes of these devices with Facedencer to feed variants of the data to the original software and see how it’s represented there.

On the bright side I feel proud of myself (maybe a little too much) for having spent the time to rewrite those two drivers with Construct while at 34C3 and afterwards. If I hadn’t refactored the code before looking at the Verio IQ, I wouldn’t have noticed the similarities so clearly and likely wouldn’t have come to the conclusion it’s a shared similar protocol. And no way I could have copy-pasted between the drivers so easily as I did.

Diabetes control and its tech: reverse engineering the OneTouch Verio

Bear with me — this post will start with a much longer trial-and-error phase than the previous one…

I have received the OneTouch Verio glucometer from LifeScan last year, when I noticed that my previous glucometer (the protocol of which was fully specified on their website) was getting EOL’d. I have used it for a couple of months, but as I posted that review, I was suggested a different one, so I moved on. It was for a while in the back of my mind as LifeScan refused providing the protocol for it, though.

So over the past week, after finishing the lower-hanging fruit I decided to get serious and figure out how this device worked.

First of all, unlike the older OneTouch devices I own, this device does not use a TRS (stereo-jack) serial port, instead it comes with a standard micro-A USB connector. This is nice as the previous cables needed to be requested and received before you could do anything at all with the software.

Once connected, the device appears to the operating system as a USB Mass Storage device – a thumbdrive – with a read-only FAT16 partition with a single file in it, an HTML file sending you to LifeScan’s website. This is not very useful.

My original assumption was that the software would use a knocking sequence to replace the mass storage interface with a serial one — this is what most of the GSM/3G USB modems do, which is why usb_modeswitch was created. So I fired the same USBlyzer (which by now I bought a license of, lacking a Free Software alternative for the moment) and started tracing. But not only no new devices or interfaces appeared on the Device Manager tree, I couldn’t see anything out of the ordinary in the trace.

Since at first I was testing this on a laptop that had countless services and things running (this is the device I used for the longest time to develop Windows software for customers), I then wanted to isolate the specific non-mass storage USB commands the software had to be sending to the device, so I disabled the disk device and retried… to find the software didn’t find the meter anymore.

This is when I knew things were going to get complicated (thus why I moved onto working on the Abbott device then.) The next step was to figure out what messages were the computer and meter exchanging; unfortunately USBlyzer does not have a WireShark export, so I had to make do with exporting to CSV and then reassembling the information from that. Let me just say it was not the easiest thing to do, although I now have a much more polished script to do that — it’s still terrible so I’m not sure I’m going to publish it any time soon though.

The first thing I did was extracting the URBs (USB Request Blocks) in binary form from the hex strings in the CSV. This would allow me to run strings on them, in the hope of seeing something such as the meter’s serial number. When reverse engineering an unknown glucometer protocol, it’s good to keep in mind essentially all diabetes management software relies on the meters’ serial numbers to connect the readings to a patient. As I’ve later discovered, I was onto something, but either strings is buggy or I used the wrong parameters. What I did find then was a lot of noise with MSDOS signatures (for MBR and FAT16) appearing over and over. Clearly I needed better filtering.

I’ve enhanced the parsing to figure out what the URBs meant. Turns out that USB Mass Storage uses signatures USBC and USBS (for Command and Status) – which also explained why I saw them in the Supermicro trace – so it’s not too difficult to identify them, and ignore them. Once I did that, the remaining URBs didn’t make much sense either, particularly because I could still notice they were only the data being written and read (as I could see many of them matched with blocks from the device’s content.)

So I had to dig further. USB is somewhat akin to a networking stack, with different layers of protocols one on top of the other — the main difference being that the USB descriptor (the stuff lsub -v prints) containing the information for all levels, rather than providing that information on each packet. A quick check on the device’s interface tells me indeed that it’s a fairly standard one:

Interface Descriptor:
  bLength                 9
  bDescriptorType         4
  bInterfaceNumber        0
  bAlternateSetting       0
  bNumEndpoints           2
  bInterfaceClass         8 Mass Storage
  bInterfaceSubClass      6 SCSI
  bInterfaceProtocol     80 Bulk-Only
  iInterface              7 LifeScan MSC

What this descriptor says is that the device is expecting SCSI commands, which is indeed the case of most USB thumbdrives — occasionally, a device might report itself as using the SDIO protocol, but that’s not very common. The iInterface = LifeScan MSC setting, though, says that there is an extension of the protocol that is specific to LifeScan. Once again here I thought it had to be some extension to the SCSI command set, so I went to look for the specs of the protocol, and started looking at the CDBs (command blocks.)

I’m not sure at this point if I was completely surprised not to see any special command at all. The only commands in the trace seemed to make sense at the time (INQUIRY, READ, WRITE, TEST MEDIA READY, etc). It was clear at that point that the software piggybacked the standard volume interface, but I expected it to access some hidden file to read the data, so I used an app to log the filesystem access and… nothing. The only files that were touched were the output Access files used by the tool.

I had to dig deeper, so I started parsing the full CDBs and looked at which part of the disk were accessed — I could see some scattered access to what looked like the partition table (but wasn’t it supposed to be read-only?) and some garbage at the end of the disk with System Volume Information. I dumped the content of the data read and written and used strings but couldn’t find anything useful, even looking for Unicode characters. So I took another trace, started it with the device already connected this time, and compared — that started sending me to the right direction: I could see a number of write-then-read requests happening on three particular blocks: 3, 4 and 5.

At that point I tried to focus on the sequence of writes and reads on those blocks, and things got interesting: some of the written and read data had the same content across sessions, which meant there was communication going on. The device is essentially exposing a register-based communication interface-over-SCSI-over-USB. I’m not sure if brilliant or crazy. But the problem remained of understanding the commands.

At this point was hoping to get some help by looking at what commands were actually being sent to the kernel, so I downloaded the latest Windows SDK and fired up WinDbg, hoping to log the events. I didn’t that, but I did find something even more interesting. The OneTouch software and drivers have been built with debug logging still on, probably because nobody would notice there is logging unless they attach a debugger… just like I did. This was a lucky breakthrough because it allowed me to see what driver the software used (and thus its symbol table and function names — yes, PE would allow you to obfuscate the function names by using an import library, but they didn’t) and also to see what it thoughts about things.

An interesting discovery is that the software seems to communicate with its drivers via XML documents (properly human-readable ones at that), while the driver seemed to talk to the device via binary commands. Unfortunately, said commands didn’t match what I was seeing in the trace, at least not fully — I could find some subsets of data here and there, but not consistently, it looks like one of the libraries is translating from the protocol the device actually accepted to another (older?) binary protocol, to speak to a driver that then converted it to XML and to the device. This does sound dopey, doesn’t it?

Anyway, I decided to then start matching messages in the sequences. This started to be interesting. Using hexdump -C to have a human-readable copy of the content of the SCSI blocks written and read, I would see the first few lines matching between messages in the same sequence, while those after 255 bytes to be different, but in a predictable way: a four-bytes word would appear at a certain address, and the following words would have the same distance from it. I was afraid this was going to be some sort of signature or cryptographic exchange — until I compared this with the trace under WinDbg, that had nothing at all after the first few lines. I then decided to filter anything after the first 16-bytes of zeros, and compare again.

This lead to more interesting results. Indeed I could see that across the three sessions, some packets would be exactly the same, while in others the written packet would be the same and the read packet would be different. And when they would be different, there would be a byte or two different and then the last two bytes would differ. Now one of the things I did when I started looking at WinDbg, was checking the symbol table of the libraries that were used by the software, and one of them had a function that included crc_ccitt in its name. This is a checksum algorithm that LifeScan used before — but with a twist there as well, it used a non-standard (0xFFFF) seed. Copying the packet up until the checksum and pasting it in an online calculator confirmed that I now found the checksum of the packet.

At that point I opened the OneTouch UltraEasy specs (an older meter, of which LifeScan published the protocol), which shared the same checksum, and noticed at least one more similarity: the messages are framed the same with (0x02 at the beginning, 0x03 at the end). And the second byte matches the length of the packet. A quick comparison with the log I got off the debugger, and the other binary protocol does not use the framing but does use the same length specification and the same checksum algo. Although in this case I could confirm the length is defined as 16-bit, as this intermediate protocol reassembled what soon clearly appeared to be a set of separate responses into one.

Once you get to this point, figuring out the commands is much easier than you think — some of them will return things such as the serial number of the device (printed on the back), the model name, or the software version, which the debug log let me match for sure. I was confused at first because strings -el can’t find them in the binary files, but strings -eb did… they are not big-endian though. At tis point, there are a few things that need to be figured out to write a proper useful driver for the meter.

The first low-hanging fruit is usually to be found in the functions to get and set time, which, given I couldn’t see any strings around, I assumed to be some sort of timestamp — but I couldn’t find anything that looked like the day’s timestamp in the trace. To be honest, there was an easier way to figure this out, but the way I did that, was by trying to figure out the reading record format. Because something that looked like a 32-bit counter in high numbers could be found, so I compared that with one that looked like it in a promising command, and I looked at the difference — the number, interpreted as seconds, gave me a 22 weeks delta, which matched the delta between the last reading and the trace — I was onto something! Given I knew the exact timestamp of the last reading, the difference between that and the number I had brought me exactly to January 1st 2000, the device’s own epoch.

Once again, from there things get easier — the format of the records is simple, includes a counter and what I soon realized to be a lifetime counter, the timestamp with the device’s own epoch, some (still unknown) flags, and the reading value in mg/dL as usual for most devices. What was curious was that the number shown in the debug log’s XML does not match the mg/dL reading, but the data in the protocol match what the device and software show for each readings, so it’s okay.

While I was working on this, I got approached over twitter from someone having a OneTouch Select Plus meter, which is not sold in Ireland at all. I asked him for a trace of the device and I compared it with my tools and the reverse engineering I had to that point, and it appears to be using the same protocol, although it replies with a lot more data to one of the commands I have not found the meaning of (and that the device don’t seem to need — there’s no knock sequence, so it’s either to detect some other model, or a kind of ping-back to the device.) The driver I wrote should work for both. Unfortunately they are both mmol/L devices, so I can’t for sure tell which unit the device is supposed to use.

One last curiosity, while comparing the protocol as I reversed it and the OneTouch UltraEasy protocol that was published by LifeScan. Many of the commands are actually matching, including the “memory reset” one, with one difference: whereas the UltraEasy commands (after preamble) start with 0x05, the Verio commands start with 0x04 — so for instance memory reset is 05 1a on the UltraEasy, but 04 1a on the Verio.

The full documentation of the protocol as I reversed it is available on my repository and glucometerutils gained an otverio2015 driver. For the driver I needed to fix the python-scsi module to actually work to send SCSI commands over the SGIO interface in Linux, but that is fixed upstream now.

If you happen to have this device, or another LifeScan device that appears as a USB Mass Storage, but using mg/dL (or something that does not appear to work with this driver), please get in touch so I can get a USB trace of its dumping memory. I could really use the help.

I won’t be spending time reverse engineering anything this weekend, because I’m actually spending time with friends, but I’ll leave you confirming that there will be at least one more device getting reverse engineered soon, but the next post will first be a review of it. The device is the Abbott FreeStyle Libre, for which I can’t link a website, as it would just not appear if you’re not in one of (the one?) country it’s sold in. Bummer.

New glucometer: LifeScan OneTouch Verio

I have visited LifeScan’s website and I noticed that their homepage is suggesting people with glucometers of the OneTouch Ultra series to upgrade to a new series. The reason for the change is purported to be found in new regulations on quality and precision of glucometers, but I could not find any reference for it.

As usual, this is a free upgrade – glucometer manufacturers are generally happy to send you a free meter, since they make most of the money on the testing strips instead – so I ordered one, and I received it (or rather picked it up) this Monday. It’s a OneTouch Verio.

The first impression is that it’s a very bad match for my previous Ultra Mini: it’s big and clunky. The whole case is about twice as big in volume. For me, as I travel a lot, this is a bit of a problem.

On the bright side, this is the first meter that I own that is not powered by CR2032 button cells. This model uses AAA batteries, which not only are much simpler to source, but are already always present in my luggage, as my noise-cancelling headphones uses the same. I also bought a 40-pack of them last time I’ve been to Fry’s.

Also, while the upgrade from OneTouch Ultra 2 to Ultra Mini maintained the same set of testing strips, the Verio comes with a new strip type. This puts it into the same bucket as the Abbott Precision Xtra in my perception, which also requires a different set of testing strips. The reason why I consider those the same is that they both have “costs” associated with the change in strips. In particular since I already have two meters (actually, three if I count the one that is set to Italian readings), which means I can keep one at home and one at the office.

Similarly to the Abbott device, and unlike the Ultras, the new strips are not coded — it makes sense since the Ultra strips also have had the same code – 25 – for the past four years. This is good because at least once every two weeks I have to draw a second sample of blood because I put the first one too soon, before the meter accepted the code. These strips also require less blood than the old ones, and even less than the Abbott strips. Funnily enough, I never got in Europe the “new” Ultra strips I’ve used in the USA before, which require about half the blood as the standard European strips. Go figure.

Also most obvious difference in these trips is that they load blood on the side rather than the tip. I’m not sure why, but I like it — it feels like it’s less likely to get ruined when you shake the strip bottle.

New meter also means new protocols of course. Unfortunately, LifeScan already refused sending me the protocol of this device, which once again puts it into the same category as Abbott’s. Interestingly enough, unlike both Abbott and the old Ultras, this device has a standard microUSB connection. Unfortunately by default it shows up as a mass-storage device with exactly one file in it: a redirect to the LifeScan website. I assume the software (which I have not installed yet) will modeswitch it to serial somehow and then proceed from there. If I had enough time I’d be setting up my Windows laptop for logging USB traffic and try to reverse engineer both this and Abbott’s protocol.

Funnily enough there is another meter, called OneTouch Verio Flex only available in Ireland, which has bluetooth connectivity. Whereas there’s the Verion Sync in the USA, which syncs with an iPhone app (no Android app, though.) I wonder if I should call them up and see if I can get myself one of those as well; it uses the same strips and would solve my problem of having more than one meter using the same stash of strips.

The meter also came with the new “OneTouch Delica” lancing system. Once again the same that was true of the meter is true of the system: it uses a new type of lancets, which is something that not even the Abbott device did. Indeed I have some four lancing devices that share the same kind of lancets already, and they even fit each other case, mostly, as they usually only have an elastic loop to fit on.

Comparatively, I’m happier about the new lancing device than I am about the meter. Not only the lancet is finer, which hurts significantly less – but would not work well with the old strips’ need of blood – but exactly as they promise on the website the new device does not vibrate, which also makes for a less painful experience.

In my experience, pricking your finger multiple times a day is the most painful side effect of the light diabetes I have, so reducing that pain has a quite significant improvement in my daily quality of life.

All in all, I’m happy that there are improvements always going on… I just wish the improvement went towards reducing the size of the device as well as improving its precision and information.