Abstract

In the pursuit of advancing the field of touch-free human-computer interaction, this paper is focused on developing a gesture enabled PC control system that aims for enhancing user engagement and providing intuitive and flexible control methods, across various applications, particularly those benefiting individuals with mobility impairments. This system has expanding potential use in virtual and augmented reality environments. This study describes a unique method for temporal gesture identification that employs gesture kinematics for feature extraction and classification. Real-time hand tracking and key point identification were performed using MediaPipe. The Euclidean distances between the key points was normalised and input into a Multilayer perceptron model, which classified the gestures and mapped them to specific commands for controlling PC functions. This approach performed well over a large dataset, improving accuracy and usability. The gesture recognition system achieved an average accuracy of 97%, with precision, recall, and F1 score of 0.924, 0.924, and 0.926, respectively, across the five gestures. This system provides the ability of customization to users which allows them to create and map their own gestures to specific commands, in addition to using predefined ones. This level of personalization and flexibility is a significant advancement over existing systems, which typically offer fixed gesture-command mappings.