THAI SIGN LANGUAGE TO ENGLISH TRANSLATION USING VARIABLE HIDDEN NEURON ANN - A CASE STUDYSowjanya M N 1 1 Assistant Professor, Department of MCA, VTU-RRC, Mysuru, Karnataka, India.2 Professor, Department of MCA, VTU-RRC, Belagavi, Karnataka, India. |
|
||
|
|
|||
|
Received 3 September 2021 Accepted 16 September2021 Published 30 September2021 Corresponding Author Sowjanya
M N, sowjanya12.mn@gmail.com DOI 10.29121/granthaalayah.v9.i9.2021.4281 Funding:
This
research received no specific grant from any funding agency in the public,
commercial, or not-for-profit sectors. Copyright:
© 2021
The Author(s). This is an open access article distributed under the terms of
the Creative Commons Attribution License, which permits unrestricted use, distribution,
and reproduction in any medium, provided the original author and source are
credited.
|
ABSTRACT |
|
|
|
Sign
language translation has been a major challenge in all walks of life. The
current society has been more accepting of the especially abled and the
government has been actively making policy changes to accommodate and assimilate
the especially abled into the society. Every country has made a conscious
effort to develop its own syllable set in its native language even though
globally used language is American Sign Language (ASL). In this paper a
method proposed by the authors for ASL is applied on Thai Sign Language and
the working of the ANN model is explored. |
|
||
|
Keywords: Sign
Language, ASL, ANN, Hidden Layer, Translation 1. INTRODUCTION The ability to
communicate to another being of the same species marked one of the greatest
advances in civilization for all animals on earth. It brought forth a new era
of languages with their own alphabet and grammar. Formal language and
linguistics have been major parts of studying the history and psychology of a
geographical area and its population. In larger countries such as India,
every state can have its own language, and this makes the study even more
granular and tedious. Genetic adversities and accidents have rendered many
people with various handicaps. The biggest of such, being inability to speak
and hear. Blindness coming in a close second, Braille Wikipedia (2021) made a
groundbreaking contribution to ease the communication in printed form.
However, the deaf and mute would have to rely on their hands and digits for
communication with other people Wikipedia (2021). A study of
various methods to aid people understand sign languages using machine
learning methods was presented in Sowjanya and
Thimmaraju (2019). In the research
work connected to this, a multiple hidden neuron model was developed to
translate sign language into English Sowjanya and
Thimmaraju (2020). The estimation
of the number of hidden neurons to provide added stability to the system was
conducted in Sowjanya and
Thimmaraju (2021). However, since
the language in discussion was ASL and the stable data set was available, the
full scope of the system could not be assessed in a comprehensive manner.
Hence the system has been applied to Thai sign language Supawadee et al.
(2012) and the results
are presented to validate the hidden neuron model for ANN based sign language
translation. The results are convincing and the system can be bench marked
against existing systems. |
|
||
2. MATERIALS AND METHODS
The dataset was taken from Supawadee et al. (2012) and Kittasil Silanon (2017). The gestures are slightly different from what is used in ASL as shown in Figure 1. Some of the initial gestures are shown in the Table 1 for reference.
|
|
|
Figure 1 ASL gestures |
The system was an ANN with 7 hidden neurons in a single layer. The system was trained with 1000 samples of gestures from TSL Kittasil Silanon (2017). Since the gestures had varying signs, the task would require 17 hours of training. The training once completed, led to the system being fed 1000 more samples for validation of the same as presented in the modern standard TSL James Woodward (1996). This took a reduced time of 9 hours. Final phase was testing. Testing was also done with 1000 samples. The results are discussed in the next section.
3. RESULTS AND DISCUSSIONS
The experiment was run on 1000 files with varying number of users as given in http://facundoq.github.io/guides/sign_language_datasets/slr. The conditions were varied in time duration of the gesture and the speed. The sample of data obtained from the results is shown below.
|
Table 1 A section of the confusion matrix generated by the translator |
||||||||||||
|
Thai gesture |
English Equivalent symbol |
No. of samples |
CM |
A |
B |
C |
D |
E |
F |
G |
H |
I |
|
A |
A |
984 |
A |
984 |
0 |
0 |
2 |
0 |
1 |
0 |
3 |
0 |
|
B |
B |
992 |
B |
0 |
992 |
0 |
0 |
2 |
0 |
0 |
0 |
0 |
|
C |
C |
974 |
C |
0 |
0 |
974 |
0 |
0 |
0 |
6 |
0 |
0 |
|
D |
D |
976 |
D |
2 |
0 |
0 |
976 |
0 |
0 |
0 |
0 |
0 |
|
E |
E |
998 |
E |
0 |
2 |
0 |
0 |
998 |
0 |
0 |
0 |
0 |
|
F |
F |
988 |
F |
1 |
0 |
0 |
2 |
0 |
988 |
0 |
0 |
0 |
|
G |
G |
985 |
G |
0 |
0 |
6 |
0 |
0 |
0 |
985 |
0 |
0 |
|
H |
H |
982 |
H |
3 |
0 |
0 |
0 |
0 |
0 |
0 |
982 |
0 |
|
I |
I |
963 |
I |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
963 |
From the data presented in Table 1, it can be seen clearly that as the number of samples increases, the accuracy improves and loss reduces drastically. The accuracy and loss plots are shown in Figure 1.
|
|
|
Figure 2 Training
and Validation parameters |
|
|
|
Figure 3 Accuracy
of translation |
The graphs clearly indicate that over time, the proposed model loss reaches 0 and remains as the translation tends to be 100% accurate.
4. CONCLUSIONS AND RECOMMENDATIONS
The proposed method having 7 neurons in the hidden layer, works well for the two most widely used sign languages. Thus, it can be extrapolated for any language and fares well against any existing system with better speed and any other user requirement.
The recommendations include that, the system can be bench marked against the existing standard translation systems and can be integrated into handheld devices such as smart phones and laptops to aid the specially abled to be a part of the society seamlessly.
REFERENCES
Sowjanya M N, Thimmaraju S N. (2019). A Comparison of Methods Used to Convert Sign Languages to Relative Formal Languages, JARDCS, Vol. 11, 01-Special Issue,
Sowjanya M N, Thimmaraju S N. (2021). Estimation of Hidden Neuron Requisite for Predictive Conversion from Sign Language to a Formal Language, WARSE, International Journal of Emerging Trends in Engineering Research, Vol 9, Issue 5, may, pp 545-548. Retrieved from https://doi.org/10.30534/ijeter/2021/02952021
Sowjanya M N, Thimmaraju S N. (2020) Multiple Hidden Neuron based model for accurate ASL translation, Sowjanya M N, Thimmaraju S N, Journal of Xi'an University of Architecture & Technology, Volume XII, Issue X, ISSN No : 1006-7930. Retrieved from https://www.xajzkjdx.cn/gallery/66-oct2020.pdf
Kittasil Silanon (2017). Thai Finger-Spelling Recognition Using a Cascaded Classifier Based on Histogram of Orientation Gradient Features Kittasil Silanon, Computational Intelligence and Neuroscience Volume, Article ID 9026375, 11 pages. Retrieved from https://doi.org/10.1155/2017/9026375
Supawadee Saengsri, Vit Niennattrakul, Chotirat Ann Ratanamahatana (2012). Thai Sign Language Translation Using Leap Motion Controller, Jirawat Tumsriand Warangkhana Kimpan, TFRS: Thai finger-spelling sign language recognition system, May. Retrieved from https://doi.org/10.1109/DICTAP.2012.6215407
Ashish S. Nikam; Aarti G. Ambekar. (2016). Sign language recognition using imagebased hand gesture recognition techniques, Online International Conference on Green Engineering and Technologies (IC-GET)
James Woodward (1996). MODERN STANDARD THAI SIGN LANGUAGE, INFLUENCE FROM ASL, AND ITS RELATIONSHIP TO ORIGINAL THAI SIGN VARIETIES, Sign Language Studies, No. 92 (Fall), pp. 227-252. Retrieved from https://doi.org/10.1353/sls.1996.0012
http://facundoq.github.io/guides/sign_language_datasets/slr
Wikipedia (2021)https://en.wikipedia.org/wiki/Braille
Wikipedia (2021) https://en.wikipedia.org/wiki/Sign_language
This work is licensed under a: Creative Commons Attribution 4.0 International License
© Granthaalayah 2014-2021. All Rights Reserved.
