Publication: Malaysian sign gesture recognition with deep learning and myo armband
Loading...
Date
2018
Authors
Nur Adrynna Nisha Mohd Rashid
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This thesis reports are a development of an improved method for hearing and speaking impaired community. There are a large scale of hearing and speaking impaired community in the whole world. They had a problem to communicate with normal people who did not know any sign language. That is why sign language is important to this community. To get a better sign language a good sign gesture is needed. In this project, the aim is to use a method of deep learning to recognize the sign gesture that be perform and translate it into text.
In previous studies, there are many methods be use to solve this problem including using different type of gesture recognition device. However, in this thesis our approach is use a friendly wearable device to recognize the Malaysian Sign Language gesture using the data that be capture from the device. The data that be capture from the device is based on two data type which are gestural data that is Electromyography and spatial data that include Accelerometer, Gyroscope and Magnetometer (orientation).
There are 12 different gesture be performed that represents many types of finger and hand gesture either in static gesture or dynamic gesture. Whereas for the sign recognition, a Long Short-Term Memory (LSTM) approach is be use which is a kind of Recurrent Neural Network (RNN) approach. After that, the dataset is preprocessed and reshaped to make sure it in the same length without have to skip any sample from the dataset. Then, LSTM will supervise the sign identification as of supervise learning in the network.
In this project, there are four way of result that will be evaluated. First one is only using the electromyography sensor data. Second way using only gyroscope sensor data. Third way is the combination of gyroscope and accelerometer sensor data. Lastly is only using accelerometer sensor data. Other than that, it will be compared with the Gated Recurrent Unit (GRU). Throughout the model implementation result that have been conducted, it shows a high accuracy of 98.3% in the electromyography sensor data. By that, it can be identified as a successful achievement approach.
Description
Q325.73.N87 2018
Keywords
Deep learning