ELECOM DEFT and the broken descriptor

Update (2017-05-26): Jiri merged the patch, which may land in 4.12 or 4.13.

In my previous post reviewing the ELECOM DEFT I noted that I had to do some work to get the three function buttons on the mouse to work on Linux correctly. Let me try to dig a bit into this one so it can be useful to others in the future.

The simptoms: the three top buttons (Fn1, Fn2, Fn3) of the device are unresponsive on Linux, they do not show up on xev and evtest.

My first guess was that they were using the same technique they do for gaming mice, by configuring on the device itself what codes to send when the buttons are pressed. That looked likely because the receiver is appearing as a big composite device. But that was not the case. After installing the Windows driver and app on my “sacrificial laptop”, and using USBlyzer to figure out what was going on, I couldn’t see the app doing anything to the device. Which meant they instead remapped the behaviour of the buttons on the software side.

This left open only the option that the receiver needs a “quirk driver” to do something. Actually, since I have looked into HID (the protocol used for USB mice and keyboards, among others), I already knew the problem was the HID Report Descriptor is reporting something broken and the Linux kernel is ignoring it. I’m not sure if Windows is just ignoring the descriptor, or if there is a custom driver to implement the quirk there. I did not look too much into this.

But what is this descriptor? If you have not looked into HID before, you have to understand that the HID protocol in USB only specifies very little information by itself, and is mainly a common format for both sending “reports” and to describe said reports. The HID Report Descriptor is effectively bytecode representing the schema that those reports should follow. As it happens, sometimes it’s not the case at all, and the descriptor itself can even be completely broken and unparsable. But that is not the case here.

The descriptor is fetched by the operating system when you connect the device, and is then used to parse the reports coming as interrupt transfer. The first byte of each transfer refers to the report used, and that is looked up in the descriptor to understand it. In most mice, your reports will all look vastly the same: state of the buttons, relative X and Y displacement, wheel (one or two dimensional) displacement. But, since the presence of one or more wheels is not a given, and the amount of buttons to expect can be significantly high, even without going to the ludicrous extent of the OpenOffice mouse, the report descriptor will tell you the size of each field in the structure.

So, looking at USBlyzer, I could tell that the report number 1 was clearly the one that gives the mouse data, and even without knowing much about HID and having not seen the report descriptor, I can tell what’s going on:

button1: 01 01 00 00 00 00 00 00
button2: 01 02 00 00 00 00 00 00
button3: 01 04 00 00 00 00 00 00
fn1:     01 20 00 00 00 00 00 00
fn2:     01 40 00 00 00 00 00 00
fn3:     01 80 00 00 00 00 00 00
Code language: HTTP (http)

So quite obviously, the second byte is a bitmask of which button is being pressed. Note that this is the first of two reports you receive every time you click on the button (and everything is zero because on a trackball you can click the buttons without even touching the ball, and so there is no movement indication in the report).

But, when I looked at the Analysis tab, I found out that USBlyzer is going to parse the reports based on the descriptor as well, showing the button number from the bitmask, the X and Y displacement and so on. For the bitmasks of the three buttons at the top of the device, no button is listed in the analysis. Bingo, we have a problem.

The quest items. Thinking of it like a quest in a JRPG, I now needed two items to complete the quest: a way to figure out what the report descriptor of the device is and what it means. Let’s start from the first item.

There are a number of ways that you find documented for dumping a USB HID report descriptor on Linux. Most of them rely on you unbinding the device from the usbhid driver and then fetching it by sending the right HID commands. usbhid-dump does that and it does well, but I’m going to ignore that. Instead I’m going to read the report descriptor as is presented by sysfs. This may not be the one reported by the hardware, but rather the one that the quirk may have already “fixed” somehow.

So how can you tell where to find the report descriptor? If you look when you plug in a device:

% dmesg | tail -n 3
[13358.058915] elecom 0003:056E:00FF.000C: Fixing up Elecom DEFT Fn buttons
[13358.059721] input: ELECOM ELECOM TrackBall Mouse as /devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2.1/3-2.1:1.0/0003:056E:00FF.000C/input/input45
[13358.111673] elecom 0003:056E:00FF.000C: input,hiddev0,hidraw1: USB HID v1.11 Mouse [ELECOM ELECOM TrackBall Mouse] on usb-0000:00:14.0-2.1/input0
% cp /sys/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2.1/3-2.1:1.0/0003:056E:00FF.000C/report_descriptor my_report_descriptor.bin
Code language: JavaScript (javascript)

You can tell from this dmesg that I’m cheating, and I’m looking at it after the device has been fixed already. Otherwise it would probably be saying hid-generic rather than elecom.

I have made a copy of the original report descriptor of course, so I can look at it even now, but the binary file is not going to be very useful by itself. But, from the same author as the tool listed above, hidrd makes it significantly easier to understand what’s going on. The full spec output includes a number of report pages that are vendor specific, and may be interesting to either fuzz or figure out if they are used for reporting things such as low battery. But let’s ignore that for the immediate and let’s look at the “Desktop, Mouse” page:

Usage Page (Desktop),               ; Generic desktop controls (01h)
Usage (Mouse),                      ; Mouse (02h, application collection)
Collection (Application),
    Usage (Pointer),                ; Pointer (01h, physical collection)
    Collection (Physical),
        Report ID (1),
        Report Count (5),
        Report Size (1),
        Usage Page (Button),        ; Button (09h)
        Usage Minimum (01h),
        Usage Maximum (05h),
        Logical Minimum (0),
        Logical Maximum (1),
        Input (Variable),
        Report Count (1),
        Report Size (3),
        Input (Constant),
        Report Size (16),
        Report Count (2),
        Usage Page (Desktop),       ; Generic desktop controls (01h)
        Usage (X),                  ; X (30h, dynamic value)
        Usage (Y),                  ; Y (31h, dynamic value)
        Logical Minimum (-32768),
        Logical Maximum (32767),
        Input (Variable, Relative),
    End Collection,
    Collection (Physical),
        Report Count (1),
        Report Size (8),
        Usage Page (Desktop),       ; Generic desktop controls (01h)
        Usage (Wheel),              ; Wheel (38h, dynamic value)
        Logical Minimum (-127),
        Logical Maximum (127),
        Input (Variable, Relative),
    End Collection,
    Collection (Physical),
        Report Count (1),
        Report Size (8),
        Usage Page (Consumer),      ; Consumer (0Ch)
        Usage (AC Pan),             ; AC pan (0238h, linear control)
        Logical Minimum (-127),
        Logical Maximum (127),
        Input (Variable, Relative),
    End Collection,
End Collection,

This is effectively a description of the structure in the reported I showed earlier, starting from the buttons and X/Y displacement, followed by the wheel and the “AC pan” (which I assume is the left/right wheel). All the sizes are given in bits, and the way the language works is a bit strange. The part that interests us is at the start of the first block. Refer to this tutorial for the nitty gritty details, but I’ll try to give a human-readable example.

Report ID is the constant we already know about, and the first byte of the message. Following that we can see it declaring five (Count = 5) bits (Size = 1) used for Buttons between 1 and 5. Ignore the local maximum/minimum in this case, as they are of course either on or off. The Input (Variable) instruction is effectively saying “These are the useful parts”. Following that it declares one (Count = 1) 3-bit (Size = 3) constant value. Since it’s constant, the HID driver will just ignore it. Unfortunately those three bits are actually the three bits needed for the top buttons.

The obvious answer is to change the descriptor so that it describe eight one-bit entries for eight buttons, and no constant bits (if you forget to remove the constant bits, the whole message gets misparsed and moving the mouse is taken as clicks, ask me how I know!). How do you do that? Well, you need a quirk driver in the Linux kernel to intercept the device, and rewrite the descriptor on the fly. This is not hard, and I know of plenty of other drivers doing so. As it happens Linux already has a hid-elecom driver, which was fixing a Bluetooth mouse that also had a wrong descriptor; I extended that to fix the descriptor. But how do you fix a descriptor exactly?

Some of the drivers check for the size of the descriptor, and for some anchor values (usually the ones they are going to change), others replace the descriptor entirely. I prefer the former, as they make it clear that they are trying to just fix something rather than discard whatever the manufacturer is doing. Particularly because in this case the fix is quite trivial, just three bytes need to be changed: change the Count and Maximum for the Buttons input to 8, and make the Count of the constant import zero. hidrd has a mode where it outputs the whole descriptor as a valid C array that you can just embed in the kernel source, with comments what each byte combination does. I used that during testing, before changing my code to do the patching instead. The actual diff, in code format, is:

@@ -4,15 +4,15 @@
 0x09, 0x01,         /*      Usage (Pointer),                */
 0xA1, 0x00,         /*      Collection (Physical),          */
 0x85, 0x01,         /*          Report ID (1),              */
-0x95, 0x05,         /*          Report Count (5),           */
+0x95, 0x08,         /*          Report Count (8),           */
 0x75, 0x01,         /*          Report Size (1),            */
 0x05, 0x09,         /*          Usage Page (Button),        */
 0x19, 0x01,         /*          Usage Minimum (01h),        */
-0x29, 0x05,         /*          Usage Maximum (05h),        */
+0x29, 0x08,         /*          Usage Maximum (08h),        */
 0x15, 0x00,         /*          Logical Minimum (0),        */
 0x25, 0x01,         /*          Logical Maximum (1),        */
 0x81, 0x02,         /*          Input (Variable),           */
-0x95, 0x01,         /*          Report Count (1),           */
+0x95, 0x00,         /*          Report Count (0),           */
 0x75, 0x03,         /*          Report Size (3),            */
 0x81, 0x01,         /*          Input (Constant),           */
 0x75, 0x10,         /*          Report Size (16),           */
Code language: JavaScript (javascript)

And that’s enough to make all the buttons work just fine. Yay! So I sent the first patch to the linux-input mailing list… and then I had a doubt “Am I the first ever Linux user of this device?” As it happens, I’m not, and after sending the patch I searched and found that there was already a patch by Yuxuan Shui sent last year that does effectively the same thing, except with a new module altogether (rather than extending the one already there) and by removing the Constant input declaration altogether, which requires a memmove() of the rest of the input. It also contains the USB ID for the wired version of the DEFT, adding the same fix.

So I went and sent another (or three) revision of the patch, including the other ID. Of course I would argue that mine is cleaner by reusing the other module, but in general I’ll leave it to the maintainers to decide which one to use. One thing that I can say at least for mine is that I tried to make it very explicit what’s going on, in particular by adding as a comment the side-by-side diff of the Collection stanza that I change in the driver. Because I always find it bothersome when I have to look into one of those HID drivers and they seem to just come up with magical constants to save the day. Sigh!