Interacting with USB HID devices from web apps

RedSpeak

Philips 2310 Foot SwitchWhile building our cloud dictation portal, RedSpeak, we ran into a limitation for web apps: hardware support. One of the main interfaces for controlling a dictation audio player while transcribing is through foot switches. Transcribers need to be able to type and control the audio player at the same time with their feet (pause, rewind one or two words, change the playback rate, etc.).

Allowing the use of USB foot switches was therefore a must-have for us. Traditionally this would have meant building platform-specific daemons to catch the USB signals and forward them to the browser, probably in the form of keyboard shortcuts. But that would mean either building many platform-specific solutions, or not offering foot switch support on all platforms. Both big no-noes for our start-up.

Instead, we turned to Chrome’s App platform and its HID (Human Interface Device) API.This API is only available for apps (so not for extensions or web scripts) and gives them direct access to any USB HID device. Our platform runs in a browser as a website, so instead of wrapping it in an app completely, we decided to build a Chrome App that would just interact with our website – as a sort of device driver if you will. This posed yet new challenges of getting the Chrome App to interact with the website in this way.

In this tutorial, we’ll show you how we did it, and explain how you can use the Chrome App platform to write a cross-platform device driver for your web apps.

1. Some background

Since Chrome 26, Chrome Apps have been able to interact with USB devices. This is pretty awesome, but not always ideal, as operating systems may take ownership of the USB device before you can even start talking to it:

Not all devices can be accessed through the USB API. In general, devices are not accessible because either the Operating System’s kernel or a native driver holds them off from user space code. Some examples are devices with HID profiles on OSX systems, and USB pen drives.

Source: Chrome Apps: USB Devices – Caveats

Luckily for us, since Chrome 38 there is a HID api as well. HID devices form a subclass of USB devices, alongside mass-storage, printers, etc. and have a simple API because they only use the control and interrupt pipes of the USB interface. Our devices are simple HID devices, so we are covered!

You can figure out whether your device is a HID device or not via the command line. On *NIX find the device first in the results of

$ lsusb

and then find it again by Bus and Device in the results of

$ lsusb -t

To verify your device is a HID devices, look for Class=Human Interface Device.

Finding information about the USB device using lsusbUsing lsusb -t to make sure your device is a HID device

One note: the HID api has changed a fair bit in Chrome 39, so we decided to support only Chrome >= 39.

2. Creating an app that interacts with a website

Our goal is to add Chrome API functionality to an existing web app (online only), so our first intuition was to simply create a content script and inject it into our app. But only Chrome Apps can get permissions to access USB devices, and content scripts are not available for apps – they are a concept from the Chrome Extensions platform.

So to inject the functionality into the web app, we needed to build a custom communication pipe between our web app and the Chrome app. We did this through Chrome’s message passing functionalities.

The gist is this: the script on the website checks if the user is running Chrome, and then tries to connect to the app. This wakes up the app, which starts listening for user input on the foot switch and sends an event to the website script if it does. The website script then decides what to do with the event.

How we'll communicate with the foot switch

2.1 Bare bones app

We first created a bare bones app, to make sure we had an app ID to work with. If you have never worked with Chrome Apps before, the Create Your First App guide from Google is a recommended primer.

We went ahead and gave our app permissions for some of the devices we were going to work with. The syntax is the following:

  "permissions": ["hid", {
      "usbDevices": [
        // Philips 2310
        { "vendorId": 2321, "productId": 6212  },
        // Olympus RS31
        { "vendorId": 1972, "productId": 607 }

        // etc...

      ]
    }
  ]

Using the lsusb command user earlier (on *NIX), you can find out the vendor and product id of your device. The manifest doesn’t accept the hexadecimal value you find there, so you need to convert it to decimal. You can quickly do this in your JavaScript console of choice with parseInt( 'hex_goes_here', 16 ).

Then, we also made sure that our domain and only our domain would be able to talk to our app, by adding it in the list of externally_connectable domains in the manifest:

"externally_connectable": {
    "matches": ["*://*.redspeak.nl/*"]
}

We installed the app as unpacked extension and made note of the App ID, so we could connect from the web script.

The whole app will live in a file called background.js. For the time being, we are just letting the app open our portal when the user starts it from the app launcher in Chrome:

chrome.app.runtime.onLaunched.addListener(function () {
  window.open('https://redspeak.nl/');
});

2.2 Web script

As preparatory work, we set up our audio player to listen for events on the window object, so future solutions can interact with our app the same way the HID device does:

$(window).on('redspeak.play',  audioplayer.play);
$(window).on('redspeak.pause', audioplayer.pause);

$(window).on('redspeak.rewind',             audioplayer.startRewind);
$(window).on('redspeak.forward',            audioplayer.startFastForward);
$(window).on('redspeak.changePlaybackRate', audioplayer.changePlaybackRate);

Then, in a separate script, we set up our pipe to the Chrome App:

// Check if the user is using Chrome
if (typeof chrome !== 'undefined'){

  // Check if the app is installed and enabled by opening a messaging port
  var port = chrome.runtime.connect('laajpojboieljeaebebfefeimjpmellm');

  if (port) {

    // When our app sends an event, it can safely be triggered on window
    port.onMessage.addListener(function(msg) {
      $(window).trigger(msg);
    });

  }

}

To recap: our web script tries to connect to our new Chrome App, and when it does, it triggers every message the app sends as an event on window. The web script is set up so that it performs the appropriate action based on events on window.

Everything is now in place to start listening to the foot switch!

2.3 Waking up the app

Message passing in Chrome Apps is quite trivial. Above, we have set up the web script to simply open a port that we can use to send and receive messages on both sides. On the app side, we just need to listen for the opening of the port:

chrome.runtime.onConnectExternal.addListener(function (port) {

  // The code here is run every time the web script on RedSpeak connects to the Chrome App.
  // This would be the moment to initialize the HID device.

  console.log( "Foot switch app has started. Initializing HID." );
  initializeHID(port);

});

That’s it for the messaging system and the initialization of the app. From the app we can now trigger events on the window object of the website by calling: port.postMessage(event).

For convenience, let’s abstract that in a function that gets the port to pass the message to as an argument:

function sendEvent(event, port) {
  port.postMessage(event);
}

For example, calling sendEvent('redspeak.play',port) from the app will make the audio player on the website start playing.

3. Interacting with the HID devices

OK! So we can now interact with the website from the app. Now for the fun part: interacting with the hardware. The workflow is as follows:

  1. Getting the list of available devices
  2. Connecting to the first available device
  3. Continuously polling the device for events
  4. Triggering an event when it happens

3.1 Connecting to the device

Step 1 is simple: chrome.hid.getDevices({},cb) passes all found devices to its callback. When we have the found devices, we can do some further checking on the devices and open the connection with the equally straight forward chrome.hid.connect(deviceId,cb).

Here is our full implementation:

function initializeHid(port) {

  // We have defined all relevant devices in manifest.json. getDevices will
  // pass all devices we can interact with to the callback. If we want to
  // filter specific devices, you can specify them as the first argument
  chrome.hid.getDevices({}, function (devices) {

    if (!devices || !devices.length) {
      console.log("No compatible device found.");
      return;
    }

    var device = devices[0];
    console.log('Found device: ', device);

    // Get a handler function that can decode the device signals
    var handler = getDeviceHandler(device);

    // Connect to the HID device
    chrome.hid.connect(device.deviceId, function (connection) {

      console.log('Connected to the HID device!');

      // Poll the USB HID Interrupt pipe
      startPoller(port, connection.connectionId, handler);

    });

  });
}

Because different foot switches give different signals, we have to use a different function to handle the signals per device. getDeviceHandler(device) should return just such a function.

For now, we just want to log all USB messages to the console. So we let getDeviceHandler return a function that decodes the ArrayBuffer it gets as arrays of unsigned integers:

function getDeviceHandler(device) {

    return function(data) {
        console.log( 'Uint8Array:', new Uint8Array(data) );
        console.log( 'Uint16Array:', new Uint16Array(data) );
        console.log( 'Uint32Array:', new Uint32Array(data) );
    };

}

With a connection to both the USB HID device and the website, we can start polling the device for signals with the call to startPoller(port, connection, handler).

3.2 Poll the device

Receiving signals from a USB device is an active process, so we need to set up a loop that continuously polls the USB device. We set up the loop with an anonymous function that is called with setTimeout like this:

function startPoller(port, connectionId, handler) {

  // The anonymous function poller() will keep the port, connectionID
  // and handler in its scope.
  var poller = function () {

    // Stop the loop as soon as the messaging port to the website
    // is no longer available.
    if (!port)
      return;

    chrome.hid.receive(connectionId, function (reportID, data) {

      var event = handler(data);
      if (event) sendEvent('redspeak.' + event, port);

      setTimeout(poller, 0);
    });

  };

  poller();
}

Hey presto, we now have an app that interacts with selected HID devices! It doesn’t do much, because we don’t have any real handlers for the devices yet, but with this scaffolding, we can start decoding device signals.

3.3 Decode device signals

Different devices return different signals, so let’s first connect one of the foot switches and listen to their signals. We start with the Philips foot switch :

Decoding the USB signals from the Philips 2310

As you can see, we can take the first value from the returned buffer when decoded as an array of unsigned 8-bit integers (make note). Trying all the buttons, on the Philips, we can map it to:

0: 'pause',   // Release
1: 'rewind',  // Left
2: 'play',    // Right
4: 'forward'  // Middle

Let’s do the same with the Olympus foot switch:

Decoding signals from the Olympus RS31

This time we need to decode the buffer as an array of 16-bit unsigned integers (again, make note for later). We can then take the 2nd value in the array and map it to actions as follows:

0: 'pause',                 // Release
1: 'forward',               // Right
2: 'play',                  // Middle
4: 'rewind',                // Left
512: 'changePlaybackRate',  // Top
514: 'changePlaybackRate'   // Middle + Top

This Olympus switch has more buttons than the Philips, so we added changePlaybackRate as an extra feature.

3.4 Handle device signals

Now that we have a straight-forward mapping of device signals to app actions, we can detect the device type and set up a signal handler accordingly. Remember that the handler was created after the port was opened, but before we started polling the device.

The final implementation looks like this:

function getDeviceHandler(device) {

    // First determine the device type
    var type = determineDeviceType(device);

    // Only allowed devices are in manifest.json, so we can safely assume
    // that we can recognize and use the device.
    console.log("Device type: ", type);

    // This function is called with the data from the HID interrupt pipe in
    // the poller function.
    return function(data) {

      if (null===data) return;

      var d = decodeSignal(data,type);

      // Events
      if (events[type] && events[type][d]) {
        return events[type][d];
      }

    };
}

Because different types of foot switches return different signals, we first determine the type in determineDeviceType(). The type returned is later used to decode the signal and then map the signal to an event.

To determine the device type, we use the same vendor and product ID as we used in manifest.json:

var types = {
  2321: {
    6212: 'philips2310'
  },
  1972: {
    607: 'olympusRS31'
  }
};

function determineDeviceType(device) {
  if (types[device.vendorId] && types[device.vendorId][device.productId]) {
    return types[device.vendorId][device.productId];
  }

  return false;
}

In decodeSignal() the buffer from the HID device is decoded according to the rules we established earlier (remember the notes we made?):

function decodeSignal(data, type) {
  var d;

  switch (type) {

    case 'olympusRS31':
      d = new Uint16Array(data);
      return d[1]; 
     
    case 'philips2310':
      d = new Uint8Array(data);
      return d[0];

  }
}

Lastly, the events are defined as an object that can easily be extended as we add more devices to the app:

var events = {
  'olympusRS31': {
    0: 'pause',                 // Release
    1: 'forward',               // Right
    2: 'play',                  // Middle
    4: 'rewind',                // Left
    512: 'changePlaybackRate',  // Top
    514: 'changePlaybackRate'   // Middle + Top
  },
  'philips2310': {
    0: 'pause',   // Release
    1: 'rewind',  // Left
    2: 'play',    // Right
    4: 'forward'  // Middle
  }
};

The app is now complete! The handler function is compiled and sent to the poller. When the poller receives data, it decodes it using these last few functions and passes the decoded message to the website script. In turn, the website script generates an event on window, which will trigger the desired effect on the audio player.

When bundling the app, you will get a new app ID, so make sure to change the app ID in the script on the website again if you are distributing a Chrome App like this. The key is based on the keypair you used to sign the package with. You can force the unpacked app to have the same key by adding the pubkey to your manifest. You can read how to do that here.

4. Wrapping up

This is all that’s needed to extend an online web-based app with extra hardware controls. The nice thing about interacting with a device through a JavaScript API is that in one fell swoop we have support for OSX and Linux, as well as all relevant Windows versions.

For us, this opens up a whole range of new opportunities. We can now develop our web app more like a desktop app, and consider interacting with Bluetooth devices (like headsets) or USB Mass Storage devices (like dictaphones).

How will you use the Chrome App platform? Let us know in the comments!

8 Comments
  • David John
    Posted at 08:33h, 25 March Reply

    This is my first post

  • Alex Rios
    Posted at 00:03h, 06 May Reply

    Very useful post! nice work guys!

  • Stuart Lemmen
    Posted at 23:04h, 18 January Reply

    Excellent post! Helped us with tough problem. Thanks.

    Please note, there is a typo/bug in the code posted. In the “initializeHid” function a call is made to the “startPoller” function with a second argument “connection”. The actual “startPoller” function listed expects a “connectionId” argument. To fix this I added this line to the “initializeHid” function just before the call to “startPoller”:

    connectionId = connection.connectionId;

    then changed the argument in that call from “connection” to “connectId”.

    -Stu

    • Mike Martel
      Posted at 09:36h, 19 January Reply

      Thanks Stuart! I’m happy to hear this post was of help. I updated the post and fixed the typo – that will save the next person some headache!

  • sakit atakishiyev
    Posted at 10:49h, 17 April Reply

    When I run this code I got this error:Uncaught TypeError: Cannot read property ‘getDevices’ of undefined, can you help me to solve

    • Mike Martel
      Posted at 12:00h, 19 April Reply

      Hi Sakit, make sure you run the code in a Chrome App that declares the correct permissions (‘hid’) in the manifest!

  • George Frueh
    Posted at 22:42h, 07 April Reply

    Hello, Don’t know if you are still taking comments with your work, but first off, thanks for sharing your work. I’m a bit new to javascript/chrome, but not new to USB development. My experience with communicating with USB devices is through VB6 and Windows API calls. I’m trying to learn from your code so that I can port my work to the Chrome environment. I’m using the https://developer.chrome.com/apps/hid website to understand how Chrome works with hid devices, but I was wondering if you might know of any other sources that dig a little deeper? I’ve modified your code to work with my USB device, and have scrolling Endpoint 1 data in the console. My next step is to place a button on the index.html page so that when I click it, I can read Endpoint 1 at that time and send the value to a text box. Any insight would be helpful! Thanks again.

    • Mike Martel
      Posted at 10:06h, 10 April Reply

      Thanks for your comment! The most helpful resource for me was the Chrome dev docs. If you have the device working with my example code, you have the difficult part behind as far as interacting with the USB device goes. The next step would be to build a solid Chrome App – you may want to look at tutorials or resources about Chrome Apps in general for that (not much out there that focuses on USB / HID devices!).

      For your app, you can either keep the code as is and keep the latest signal in a local variable, that you write to the input field when the button is clicked. Alternatively, you can remove poller, and only call `chrome.hid.receive` when the button is clicked and print the next result in the input field.

Post A Reply to Stuart Lemmen Cancel Reply