Real-Time Hand Gesture Interface for PC Control Using Computer Vision

Abstract

This project presents a real-time hand gesture recognition system designed to enable touchless human-computer interaction using a webcam. Leveraging Me diaPipe for hand landmark detection and TensorFlow for gesture classification, the system processes video frames to extract normalized keypoints, which are fed into a trained LSTM-based model (with an advanced variant incorporating Conv1D and Attention mechanisms) to classify gestures such as swipes, waves, and pinches. These gestures are mapped to computer actions like key presses and mouse movements via pyautogui, facilitating applications in browser navigation, media control, and accessibility. The project encompasses data collection, preprocessing, model training, and performance evaluation, with metrics visualized through plots of accuracy, loss, and F1 scores. Configurable through a centralized JSON file, the system achieves robust performance with a confidence threshold of 0.8 for most gestures. Results demonstrate effective real-time gesture recognition, with potential for further optimization in gesture coverage and dataset consistency