Paired with a custom AI algorithm, the device translates these ultrasound images into the precise positions of the five fingers and palm across 22 degrees of freedom. It was successfully tested on eight volunteers with different hand sizes, who performed gestures including all 26 American Sign Language letters, grasping objects (e.g., tennis ball, bottle, scissors, pencil), and manipulating virtual objects on a screen.
The wristband enables wireless control of a robotic hand, allowing users to make it perform dexterous tasks such as playing a piano tune or shooting a basketball. Researchers highlight that it offers a more natural and accurate alternative to existing hand-tracking methods like cameras or sensor gloves, which can be cumbersome or imprecise. Potential applications include intuitive controls for virtual/augmented reality and generating large datasets of human hand motions to train dexterous humanoid robots (e.g., for surgery or fine manipulation).
The work, led by Xuanhe Zhao and colleagues, was published in Nature Electronics.
