About This Project
This project introduces an eye-controlled mouse system that works with a fun bee game. The goal is to create a simple and affordable way for people, especially those with physical disabilities, to interact with computers. The system uses a webcam to track eye movements, moving the mouse pointer and clicking based on where you look. The bee game shows how this system can be used for both practical tasks and games.
Introduction
Computers play a big role in our daily lives, but not everyone can use a mouse or keyboard easily. This is especially true for people with motor disabilities who may face significant challenges using traditional input devices. Eye-controlled systems offer a promising solution, allowing users to interact with a computer simply by moving their eyes. However, many of these systems are either expensive or too complicated for everyday use.
This project focuses on creating a low-cost, user-friendly eye-controlled mouse system using widely available hardware like a webcam and free, open-source software. The system is designed to be accessible to everyone and can be used for both serious tasks and entertainment. To demonstrate its potential, we included a simple bee game where players control a bee by moving their eyes and blinking to collect nectar. This fun addition shows that the system can be applied in creative and engaging ways.
By combining accessibility with ease of use, this project aims to make technology more inclusive for people with disabilities while also inspiring new uses for eye-tracking technology.
Key Features
- Develop a hands-free computer mouse system based on eye tracking.
- Leverage Mediapipe's Face Mesh for accurate detection of facial landmarks.
- Implement real-time cursor movement and clicking using specific eye gestures.
- Implement real-time cursor movement and clicking using specific eye gestures.
- Ensure ease of use, accessibility, and reliable performance across various lighting conditions.
Methodology
The methodology involves integrating accessible hardware with powerful software tools to create a reliable and intuitive system.
The system’s hardware consists of a standard webcam to capture the user’s facial movements and a computer to process the data. The software framework includes three primary libraries: OpenCV for image processing, MediaPipe for detecting facial landmarks, and PyAutoGUI for simulating mouse actions.
The workflow begins with the webcam capturing real-time video of the user. MediaPipe then processes this video to identify facial landmarks, focusing on the eyes and blinks. The system calculates the user’s gaze position and translates it into mouse pointer movements. Blinks are detected as mouse clicks. These functionalities are further applied in the bee game, where users control the bee’s movements and actions, such as collecting nectar.
This combination of hardware and software ensures affordability, ease of use, and wide applicability across different settings.
The system is implemented using Python with the following tools:
- OpenCV: For video capture and processing.
- Mediapipe: For face mesh detection and landmark extraction.
- PyAutoGUI: For cursor control and mouse click simulation.
The webcam captures video frames, which are processed to extract RGB images for Mediapipe. The facial landmarks are scaled to match screen dimensions, enabling cursor movement. Intentional blinks are detected using the pre-defined thresholds and trigger mouse clicks.
Results
The Eye-Controlled Mouse System successfully demonstrated its capability to provide precise and responsive cursor control using eye movements. The integration of Mediapipe's Face Mesh for facial landmark detection allowed the system to track eye positions with high accuracy. Cursor movements were smooth, with minimal latency, as the facial landmarks were normalized and mapped to screen coordinates effectively.
The blink detection mechanism, which calculated the vertical distance between two specific eye landmarks (145 and 159), proved reliable in distinguishing natural blinks from intentional commands. This enabled the system to simulate mouse clicks seamlessly. Users could perform click operations with intentional eye blinks, offering a hands-free and efficient alternative to traditional mouse usage.
Furthermore, the system was tested across different screen resolutions and sizes, adapting well to various configurations without compromising performance. The visual feedback provided on the screen, where key landmarks were highlighted in real time, ensured that users could adjust their movements for optimal control. The system's robustness and adaptability in real-time scenarios highlight its potential as a practical and accessible tool for hands-free interaction.
Contributors
- Piyamini de Saram
- Akalanka Vimukthi
- Nimesha Sewwandi