Regresar

Augmented Reality for Learning Sign Language Using Kinect Tool

Abstract:

The use of Augmented Reality (AR) has increased markedly as an educational tool due to the implementation of smart devices (PCs, smartphones or tablets), that complement the interaction with the real world, allowing a better perception of the environment for people with hearing impairment by the additional information it generates. This article focuses on developing a low-cost functional application that is interactive and easy to manipulate for people with hearing disabilities or people who work or socialize with them, with the aim of facilitating the process of learning sign language. The implementation was carried out using Augmented Reality along with the Kinect V2 device, due to the fidelity of the depth camera and RGB camera, which enable the design of the learning and evaluation process using Unity software. Finally, the results obtained by carrying out the experimental method and performing the application’s operational tests for prototype validation are presented, achieving a gesture recognition accuracy of 90%. In this way, a reinforcement and complement is allowed in the learning of Ecuadorian sign language in an interactive way, providing an improvement in the quality of life of people with hearing impairment.