MouseRing: Always-available Touchpad Interaction with IMU Rings
Keyword: Ubiquitous computing, IMU sensing, Touch Interface, Physical Modeling
We propose MouseRing, a novel ring-shaped IMU device that enables continuous finger-sliding on unmodified physical surfaces like a touchpad. Tracking fine-grained finger movements with IMUs for continuous 2D-cursor control poses significant challenges due to limited sensing capabilities. Our work suggest that finger-motion patterns and the inherent structure of joints provide beneficial physical knowledge, which leads us to enhance motion perception accuracy by integrating physical priors into ML models. A motion dataset was created using infrared cameras, touchpads, and IMUs. We then identified several useful physical constraints, such as joint co-planarity, rigid constraints, and velocity consistency. These principles help refine the finger-tracking predictions from an RNN model. By incorporating touch state detection as a cursor movement switch, we achieved precise cursor control. In a Fitts’ Law study, MouseRing demonstrated input efficiency comparable to touchpads. In real-world applications, MouseRing ensured robust, efficient input and good usability across various surfaces and body postures.
Target selection is one of the most fundamental tasks in human-computer interaction. Traditional methods, such as mouse and touchpad, have been widely adopted due to their intuitive design, efficiency, and precision. However, their physical attributes present certain constraints, particularly in mobile environments. The rise of ubiquitous computing devices such as AR/VR and large-screen displays has generated demands for always-available input solutions. While remote controllers and computer-vision-based finger-pointing techniques have found their niche, their complex setup and large computational power requirements restrict them from being always accessible.
We propose that wearable IMU rings potentially serve as an always-available touch interface. Our goal is to retain the efficiency and comfort characteristic of touchpad interactions while optimizing the device’s form factor to enhance its portability and availability at all times. IMU rings are small and low-powered compared to cameras or electromagnetic sensors, making them suitable for long-term daily use. However, precise finger tracking based on IMU rings is challenging due to limited information and noisy signals. While researchers have attempted physical mapping and machine learning methods to solve gesture classification and typing tasks, few prior works have demonstrated the use of IMU rings for high-precision 2D cursor control and target selection tasks.
We propose MouseRing, a ring-formed IMU device that can accurately track fingertip movement trajectories. By incorporating touch state detection as a cursor movement switch, MouseRing enables always-available touchpad
interaction. In our work, we follow a data-driven research process. We first uncover some physical constraints as prior during finger-sliding based on data analysis. We then train ML models for fingertip velocity prediction. Finally, we integrate the knowledge of both for more stable and accurate tracking.
We construct a multimodal motion dataset of sliding fingers using OptiTrack, pressure touchpads, and IMUs. The dataset contains contact points, nail tips, and several key joints of the index fingers. Following the research approaches in computer vision, we model the index finger as articulated objects in 3D space with interconnected joints. We propose several hypotheses that satisfy certain constraint relationships. By analyzing the data, we verify the existence of co-planarity, velocity correlation, rigid constraints, and other physical relationships between joint nodes. We use an end-to-end RNN model to predict the instantaneous velocity of fingertips. We then quantify the degree of conformity between the predicted velocity and the physical constraints with confidence scores. The confidence score serves as a weight to correct and smooth the trajectory. Using the dataset for offline simulation, we compare the performance of different model settings, ring numbers, and sliding modes. In a purely kinematics-based baseline model, the tracking error is significant. End-to-end models shows acceptable performance. By incorporating physical constraints, the results become more stable and accurate. Dual MouseRing can accurately select small targets, with a mean angular error of 6.6◦, while a single MouseRing has a mean angular error of 12.3◦ but is more lightweight.
We evaluate the input efficiency of MouseRing in ideal laboratory conditions and real-world situations. In the Fitts’ Law Study, MouseRing achieves input efficiency close to laptop touchpads (MT = 658.5ms vs. 629.1ms). In a real-world large-screen interaction task, single and dual MouseRing can both achieve robustly and quickly 2D cursor control on surfaces of different hardness and flatness and in standing and sitting postures. Its speed is similar to that of mouse devices and significantly outperforms AirMouse devices, which share the same in-air interaction paradigm as visual hand tracking and remote controllers. Participants appreciated the naturalness of the interaction and found wearing MouseRing more comfortable than hand-held devices.