Electronics meets Usability: The Hands-Free Mouse
🌱Empowering Accessibility
Cerebral palsy (CP) affects the lives of many individuals, impairing their muscle coordination and movement. It is the most common motor disability among children and often makes using traditional computer input devices, such as a mouse, a challenging task. In this article, we explore the development and testing of a hands-free mouse designed to empower individuals with cerebral palsy by enabling them to use computers without manual dexterity. This innovative solution leverages real-time facial expression and head movement tracking to control the cursor. The results of our experiments indicate that the hands-free mouse can serve as a viable alternative to conventional computer input devices, potentially enhancing the quality of life for individuals with cerebral palsy. While this technology shows great promise, further refinement is needed to improve accuracy and user experience. Moreover, it opens the door to exploring its potential benefits for people with other motor disabilities.
🌟 The Need for Innovation
Cerebral palsy (CP) encompasses a range of neurological disorders affecting an individual’s movement, balance, and posture, primarily impacting children. The fine motor skill challenges posed by CP can make using traditional computer input devices, like a mouse, an uphill battle, limiting one’s participation in academic, professional, and social activities. This underscores the pressing need for a hands-free mouse designed to make computer access possible for those with CP.
Studies have shown that assistive technology devices can significantly improve computer access for individuals with CP. For example, head-mounted and foot-controlled mouse devices have demonstrated their effectiveness in enhancing computer access. However, they often require additional physical effort, which might not be feasible for individuals with severe motor impairments.
A hands-free mouse presents a promising alternative, offering a more natural and comfortable interaction with computers for those with CP. A study by Nasor and colleagues developed a hands-free mouse using eye-tracking technology, showing its efficacy in improving computer access and receiving high user satisfaction. However, the model has limitations, such as lag. This article introduces an innovative hands-free mouse solution based on real-time facial expression and head movement tracking.
🔬 The Cutting-Edge Solution
Our hands-free mouse system leverages a computer, a webcam, the dlib C++ library, and the OpenCV library for real-time facial feature point extraction and face orientation estimation. The Dlib library is open-source and capable of extracting 68 facial feature points from the face area of an image, enabling OpenCV to determine the face’s orientation. This technology provides three essential functions:
1. Activation and deactivation of mouse control through mouth state recognition.
2. Control of the mouse pointer based on face direction.
3. Mouse clicking by detecting open or closed eyes.
To recognize mouth state, we extract eight mouth feature points, calculate the Mouth Aspect Ratio (MAR), and use a threshold value (Th_m) to determine if the mouth is open or closed. The face’s orientation is determined by the two-dimensional placement of six key facial feature points. The mouse cursor is controlled using the PyAutoGUI package.
The system relies on prebuilt models for face detection and landmark prediction. A face detector is created using the Histogram of Oriented Gradients (HOG) feature, a linear classifier, image pyramid, and sliding window detection approach. The landmark predictor is trained on the iBUG 300-W face landmark dataset [6]. For eye aspect ratio (EAR) calculation, we use eye landmarks and set a threshold for blink detection.
🔍 Testing the Waters
In a series of experiments, participants were tasked with specific challenges to evaluate the hands-free mouse’s effectiveness. The participants included a 7-year-old male and a 13-year-old female with cerebral palsy, and a 14-year-old male with intellectual disabilities. Each experiment assessed the device’s capability to complete specific tasks.
1. Experiment 1: Moving the cursor along a circle with a 2-inch radius.
2. Experiment 2: Scrolling to the end of a page using the scroll mode.
3. Experiment 3: Printing a document by right-clicking and selecting the “Print” option.
The time taken to complete each task and the success rate were recorded for all participants.
🙌 Outcomes That Matter
The results of the experiments revealed the hands-free mouse’s viability for individuals with cerebral palsy and intellectual disabilities. In Experiment 1, the hands-free mouse exhibited precise cursor movements, with an average completion time of 1.3 minutes. In Experiment 2, the device successfully scrolled to the end of a page in an average of 2.7 minutes. Experiment 3, the most challenging of the three, involved printing a document, with an average completion time of 5.3 minutes. The hands-free mouse was able to complete each task with an acceptable level of accuracy.
The participants experienced a learning curve, but their accuracy improved with practice. This indicates that users can adapt to the interface, ultimately achieving greater accuracy and efficiency.
🌎 Shaping a More Inclusive World
The hands-free mouse developed for individuals with cerebral palsy has demonstrated its potential as an effective alternative to traditional computer input devices. Our experiments indicated that this innovative technology reduced completion times for various tasks, enhancing efficiency and productivity.
The positive feedback and satisfaction reported by participants emphasize the user-friendliness and comfort of this device, particularly important for individuals with motor disabilities. While further research is needed to enhance accuracy and reduce errors, the hands-free mouse is a significant step toward creating a more inclusive and accessible world for individuals with disabilities. This technology opens doors to broader opportunities in education, work, and daily life, significantly improving the quality of life for those with motor disabilities.
