Video Conferencing with Gesture Based AR Control

Abstract

Video conferencing in this day and age has been a norm due to an unforeseen pandemic that halted many daily activities. It can be seen that many video conferencing applications which were previously not so familiar to the general public came into popularity viz. Zoom and Microsoft Teams. The present video conferencing applications are doing fine for the current scenario as it has helped millions of people to connect. However, video conferencing will be more attractive with the addition of augmented reality and gesture-based control system. Adding of augmented reality to video conferencing make presentations more interactive. It is suggested that augmented reality is an intuitive and user-friendly paradigm to communicate information about the physical environment. With the addition of augmented reality, users get an alternative and interactive way of presenting themselves during a conference. In addition to gesture control, this project includes the controlling of the augmented reality items and other features of the video conference via hand gestures.For the implementation of the hand gesture control of the augmented reality, library provided by google, mediapipe hands(consisting of palm detection model and hand landmark model) is used and to control the basic functions of the application such as toggling the microphone, opening and closing chat bar etc., model made from video vision transformer is used.The introduction of the hand gesture removes the bottleneck in the effective utilization of the available information flow as human already uses lots of hand gestures to communicate non verbally.