Just from patterns of motion, your smart devices know when youâre walking, when youâre riding a bike, and when you lift your wrist to check the time. But it turns out they can also tell when you snap or make a fist, or whether youâre holding a smartphone or steering wheel. All they have to do is listen a little harder â well, about 100 times harder, actually.
Researchers from Carnegie Mellon University created a system called ViBand that supercharges an ordinary smartwatchâs accelerometer, allowing it to sense incredibly tiny variations in vibration frequency. That could be the thrum of an engine, the note being tuned on a guitar, or the slight differences apparent when you move your hand in different ways.
The secret lies in the specs of the accelerometer itself. Normally, they sample motion somewhere around 20-100 times per second â more than enough to tell whether the user is walking or running, for instance.
But CMUâs Chris Harrison and his colleagues noticed something.
âRight there on the data sheet, it says âmaximum speed 4,000 Hz,’â said Harrison in an interview with TechCrunch. It was capable of polling motion more than a hundred times faster than any smartphone was telling it to. âWe saw that and said hmm, bet thereâs some interesting stuff there.â
Sure enough, there was. Even when propagated through âa water-filled sack of bones,â as Harrison described the body, just about everything produces a unique high-frequency vibration pattern, a sort of acoustic signature that can be used to identify it almost immediately.
At first, the team considered use cases such as being able to snap your fingers to turn on a light. âBut that seemed kind of gimmicky,â he said. âMore interesting is being able to use the arm as an extension of this sensor. We can actually detect what object youâre grasping as soon as you grasp, and we can detect whether youâre in the car, in the kitchenâ¦â
Think of a timer that turns on as soon as you grab your toothbrush, maps that appear when you touch the map at the entrance of a building, or a 2-factor system that checks not just whether you have a device, but whether youâre sitting at your desk. It can be combined with active electrical and wireless signals sent out by the watch to strengthen the recognition process.
âVarious people in the industry have reached out to us; theyâre like, âhuh, didnât know we could do this.â Weâre in talks with some people right now,â said Harrison. âThe capabilities we show, Apple or Samsung or whoever got on board could deploy it â itâs all software.â
âI mean, thereâs a reason we hacked an Android watch,â he added. âAll these watches have accelerometers and they all have high speed modes. But we couldnât do it on an Apple watch, itâs very hard to hack.â
This isnât the first foray into augmenting the capabilities of smartwatches made by Harrisonâs lab. Other work has shown the possibility of wirelessly detecting the position of a finger nearby the watch, or on the skin of the arm and hand the watch is attached to.
âSmartwatches have to have like a one-inch screen,â Harrison said, âso how do you expand the envelope of interaction without sticking a giant screen on it? Thatâs research weâve been working on for about five years.â
The teamâs work was selected to receive one of four âbest paperâ awards at the Association for Computing Machineryâs User Interface Software and Technology Symposium in Tokyo.