Publisher:Institute of Advanced Engineering and Science (IAES)
Series Info:Indonesian Journal of Electrical Engineering and Computer Science;Vol. 30, No. 1, April 2023, pp. 545~556
Type:Article
Keywords:Action detection
,
Hand gesture
,
LSTM model
,
MediaPipe
,
Sign language
Abstract:
Sign language recognition is very important for deaf and mute people because
it has many facilities for them, it converts hand gestures into text or speech. It
also helps deaf and mute people to communicate and express mutual feelings.
This paper's goal is to estimate sign language using action detection by
predicting what action is being demonstrated at any given time without forcing
the user to wear any external devices. We captured user signs with a webcam.
For example; if we signed “thank you”, it will take the entire set of frames for
that action to determine what sign is being demonstrated. The long short-term
memory (LSTM) model is used to produce a real-time sign language detection
and prediction flow. We also applied dropout layers for both training and
testing dataset to handle overfitting in deep learning models which made a
good improvement for the final result accuracy. We achieved a 99.35%
accuracy after training and implementing the model which allows the deaf and
mute communicate more easily with society