Voice to Sign Language Translation Using Speech Recognition

Abstract

Sign Languages are used by deaf people as they way the communicate. Unlike sound language, sing language is a visual language. Which it uses combination of arms and facial expression and hands movement to express they want to say. Sign language used by the deaf people who born deaf which has disability to speak or by people can hear but cannot speak or by normal people who wants to communicate with disabled people. So, for a deaf person it is very important to have access to a sign language for their emotional, linguistic growth and social. This project aims to fill the gap between disabled people and normal people or make the disabled to live their live easy, by using deep learning and natural language processing techniques. The main purpose of the project is to bult a system that take the input in form of sound wave (audio) then converting it to sign language. This achieved by using 2 phases. The first is speech recognition which take the audio file and extract the MFCC features and pass it to trained model that used Dilated Convolution network to pass it to CTC Loss to predict the input and convert it to text. Second phase is applying the Semantics of Natural Language Processing (Tokenize and porter stemmer) for making the input is suitable for American sign language grammar which pass to database contains videos of each sign then merge the input into one output video.