Home   >   CSC-OpenAccess Library   >    Manuscript Information
Full Text Available

(2.13MB)
This is an Open Access publication published under CSC-OpenAccess Policy.
Publications from CSC-OpenAccess Library are being accessed from over 74 countries worldwide.
Video Audio Interface for recognizing gestures of Indian sign Language
E.Kiran Kumar, S.R.C.Kishore , P.V.V.Kishore, P.Rajesh Kumar
Pages - 479 - 503     |    Revised - 01-09-2011     |    Published - 05-10-2011
Volume - 5   Issue - 4    |    Publication Date - September / October 2011  Table of Contents
MORE INFORMATION
KEYWORDS
Sign language, Wavelet transform, Image Fusion, Elliptical Fourier Descriptors, Principle Component Analysis, Fuzzy Inference System
ABSTRACT
We proposed a system to robotically recognize gestures of sign language from a video stream of the signer. The developed system converts words and sentences of Indian sign language into voice and text in English. We have used the power of image processing techniques and artificial intelligence techniques to achieve the objective. To accomplish the task we used powerful image processing techniques such as frame differencing based tracking, edge detection, wavelet transform, image fusion techniques to segment shapes in our videos. It also uses Elliptical Fourier descriptors for shape feature extraction and principal component analysis for feature set optimization and reduction. Database of extracted features are compared with input video of the signer using a trained fuzzy inference system. The proposed system converts gestures into a text and voice message with 91 percent accuracy. The training and testing of the system is done using gestures from Indian Sign Language (INSL). Around 80 gestures from 10 different signers are used. The entire system was developed in a user friendly environment by creating a graphical user interface in MATLAB. The system is robust and can be trained for new gestures using GUI.
CITED BY (18)  
1 Joudaki, S., Mohamad, D. B., Saba, T., Rehman, A., Al-Rodhaan, M., & Al-Dhelaan, A. (2014). Vision-Based Sign Language Classification: A Directional Review. IETE Technical Review, 31(5), 383-391.
2 Melnyk, M., Shadrova, V., & Karwatsky, B. (2014). Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey. International Journal of Computer Applications, 89(17), 44-51.
3 Babu, S. S., Subrahmanyam, K. V., & Tech, M. Lung Cancer Diagnosis using PET/CT Images based on preprocessing methods.
4 Venkatram, N., reddy, l., kishore, p., & shavya, c. (2014). rsa-dwt based medical image watermarking for telemedicine applications. Journal of Theoretical & Applied Information Technology, 65(3).
5 Gimbi, T. (2014). Recognition of Isolated Signs in Ethiopian Sign Language (Doctoral dissertation, Addis Ababa University).
6 Venkatram, N., Reddy, L. S. S., & Kishore, P. V. V. (2014). Multiresolution medical image watermarking for telemedicine applications. Digital Image Processing, 6(1), 6-15.
7 Futane, P. R., Dharaskar, R. V., & Thakare, V. M. (2014). Survey and analysis of Indian Sign Language Recognition research. International Journal of Advanced Research in Computer Science, 5(4).
8 Divya, S., Kiruthika, S., & Padmavathi, S. Segmentation, Tracking And Feature Extraction For Indian Sign Language Recognition.
9 Venkatram, N., Reddy, L. S. S., Kishore, P. V. V., Fields, G., Vaddeswaram, G. D., & Pradesh, A. (2014). Blind Medical Image Watermarking with LWT–SVD for Telemedicine Applications. image, 20, 23.
10 Venkatram, N., Reddy, L. S. S., & Kishore, P. V. V. Dwt-Bat Based Medical Image Watermarking For Telemedicine Applications.
11 Kishore, P. V. V., Venkatram, N., Sarvya, C., & Reddy, L. S. S. (2014, August). Medical image watermarking using RSA encryption in wavelet domain. In Networks & Soft Computing (ICNSC), 2014 First International Conference on (pp. 258-262). IEEE.
12 Reshna, S., & Jayaraju, M. (2015). A survey on Segmentation and Feature extraction of Images in Indian Sign Language Recognition System.
13 Nair, A. V., & Bindu, V. (2013). A Review on Indian Sign Language Recognition. International Journal of Computer Applications, 73(22), 33-38.
14 Kishore, P. V. V., Rahul, R., Sravya, K., & Sastry, A. S. C. S. (2015, August). Crowd Density Analysis and tracking. In Advances in Computing, Communications and Informatics (ICACCI), 2015 International Conference on (pp. 1209-1213). IEEE.
15 Ghotkar, A. S., & Kharate, G. K. (2015). Dynamic Hand Gesture Recognition and Novel Sentence Interpretation Algorithm for Indian Sign Language Using Microsoft Kinect Sensor. Journal of Pattern Recognition Research, 1, 24-38.
16 Kausar, N., Samir, B. B., & Kuleev, R. (2006). Lung cancer detection using supervised classification with cluster variability on radiographs data. women, 25, 19-04.
17 Savitha, S. K., & Naveen, N. C. Study for Assessing the Advancement of Imaging Techniques in Chest Radiographic Images.
18 Kishore, P. V. V., Mallika, K. L., Prasad, M. V. D., & Narayana, K. L. (2015). Denoising Ultrasound Medical Images with Selective Fusion in Wavelet Domain. Procedia Computer Science, 58, 129-139.
1 Google Scholar 
2 CiteSeerX 
3 refSeek 
4 Scribd 
5 SlideShare 
6 PdfSR 
1 W. Stokoe, D. Casterline, and C. Croneberg. “A Dictionary of American Sign Language on Linguistic Principles.” Gallaudet College Press, Washington D.C., USA, 1965.
2 U. R. Wrobel. Referenz in eb¨ardensprachen: Raum und Person. Forschungsberichte des Instituts f¨ur Phonetik und Sprachliche Kommunikation der Universit ¨at M¨unchen, 37:25– 50, 2001.
3 C. Vogler and D. Metaxas. “A framework for recognizing the simultaneous aspects of american sign language”. Computer Vision & Image Understanding, 81(3):358–384, March 2001.
4 Ruiduo Yang, Sudeep Sarkar, and Barbara Loeding. Enhanced level building algorithm to the movement epenthesis problem in sign language. In “CVPR”, MN, USA, June 2007.
5 ASL corpus: .
6 Christopoulos, C., Bonvillian, J., 1985. “Sign language”. Journal of Communication Disorders 18, 1–20.
7 Atherton, M., 1999. Welsh today BSL tomorrow. In: Deaf Worlds 15 (1),pp. 11–15.
8 Engberg-Pedersen, E., 2003. From pointing to reference and predication: pointing signs, eyegaze, and head and body orientation in Danish Sign Language. In: Kita, Sotaro (Ed.), Pointing: Where Language, Culture, and Cognition Meet. Erlbaum, Mahwah, NJ,pp. 269– 292.
9 Nyst, V., 2004. Verbs of motion in Adamorobe Sign Language. Poster. In:TISLR 8 Barcelona, September 30–October 2. Programme and Abstracts. (Internat. Conf. on Theoretical Issues in Sign Language Research; 8), pp. 127–129.
10 Abdel-Fattah, M.A., 2005. Arabic sign language: a perspective. Journal of Deaf Studies and Deaf Education 10 (2), 212–221.
11 Masataka, N. et al., 2006. Neural correlates for numerical processing in the manual mode. Journal of Deaf Studies and Deaf Education 11 (2), 144–152.
12 Indian Sign language, empowering the deaf, Indian Sign Language Database .
13 Gaolin Fang and Wen Gao, “Large Vocabulary Contineous Sign languagre Recognition Based on Trnsition-Movement Models”, IEEE Transaction on Systems,MAN, and Cybernetics-Vol.37,No.1,January 2007,pp 1-9.
14 T.Starner and A.Pentland “Real-Time American Sign Language Recognition from video using Hidden Markov Models”,Technical Report, MIT Media laboratory Perceptual computing section, Technical Report number.375,1995.
15 Ming-Hsuan Yang and Narendra Ahuja, “ Extraction of 2D Motion Trajectories and its aApplication to Hand Gesture Recognition”, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.24, No.8, August 2002,pp1061-1074.
16 M.K.Bhuyan and P.K.Bora, “A Frame Work of Hand Gesture Recognition with Applications to Sigb Language”, Annual India Conference, IEEE, pp1-6.
17 Rini Akmeliawati, Melanie Po-Leen Ooi and Ye Chow Kuang , ‘Real-Time Malaysian Sign Language Translation Using Colour Segmentation and Neural Network’, IEEE on Instrumentation and Measurement Technology Conference Proceeding,Warsaw, Poland 2006 ,pp. 1-6.
18 Nariman Habili, Cheng Chew Lim and Alireza Moini , ‘Segmentation Of The Face And Hands In Sign Language Video Sequences Using Color And Motion Cues’, IEEE Transactions on Circuits and Systems For Video Technology 2004 , Vol. 14, No. 8, , pp.1086 – 1097
19 Yu Zhou and Xilin Chen, “Adaptive sign language recognition with Exemplar extraction and MAP/IVFS”, IEEE signal processing letters, Vol 17, No-3, March 2010, pp297-300.
20 Och J., Ney. H., 2002. Discriminative training and maximum entropy models for statistical machine translation. In: Annual Meeting of the Ass. For Computational Linguistics (ACL), Philadelphia, PA, pp. 295–302.
21 Sumita, E., Akiba, Y., Doi, T., et al., 2003. A Corpus-Centered Approachto Spoken Language Translation. Conf. of the Europ. Chapter of the Ass. For Computational Linguistics (EACL), Budapest, Hungary, pp. 171–174.
22 Casacuberta, F., Vidal, E., 2004. Machine translation with inferred stochastic finite-state transducers. omputational Linguistics 30 (2),205–225.
23 Och, J., Ney, H., 2003. A systematic comparison of various alignment models. Computational Linguistics 29 (1), 19–51.
24 Koehn, P., 2004. Pharaoh: a beam search decoder for phrase-based statistical machine translation models. AMTA.
25 ESIGN project: .
26 ECHO corpus: .
27 Liu Cai, “a kind of advanced Sobel image edge detection algorithm”, Guizhou Industrial College Transaction (Natural Science Edition),2004, 33(5):77-79.
28 Baris Sumengen, B. S. Manjunath “ multi-scale edge detection and image segmentation”, ece dept. UC, santa Barbara, USA,(National Conference)2005.
29 Lily Rui, Liang, Carl G., Looney, “Competitive Fuzzy Edge Detection”, Applied Soft Computing, Vol.3, no.4, pp. 123-137, 2003.
30 G. Pajares, “A Wavelet-based Image Fusion Tutorial”, Pattern Recognition, Vol. 37, no.10, pp. 1855-1872, 2004.
31 J.N. Ellinas, M.S. Sangriotis, “Stereo Image Compression Using Wavelet Coefficients Morphology”, Image and Vision Computing, Vol.22, no.2, pp. 281-290, 2004.
32 Kuhl, F. P. and Giardina, C. R., Elliptic Fourier Descriptors of a Closed Contour, CVGIP, 18,pp. 236–258, 1982
33 Lin C. C. and Chellappa, R., Classification of Partial 2D Shapes using Fourier Descriptors,IEEE Trans. PAMI, 9(5), pp. 686–690, 1987.
34 Montiel, E., Aguado, A. S. and Zaluska, E., Fourier Series Expansion of Irregular Curves, Fractals, 5(1), pp. 105–199, 1997.
35 Persoon, E. and Fu, K.-S., Shape Description Using Fourier Descriptors, IEEE Trans. SMC, 3, pp. 170–179, 1977
36 M. Sugeno, “An introductory survey of fuzzy control,” Inform. Sci., vol. 36, pp. 59-83, 1985.
37 C. C. Lee, “Fuzzy logic in control systems: Fuzzy logic controller-Part-I and Part-II,”IEEE Trans. Syst., Man, Cybem., vol. 20, no. 2, pp. 404-435, 1990.
38 T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Trans. Syst., Man, cybern., vol. 15, no. 1, pp. 116-132, 1985.
39 L. X. Wang and I. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Trans. Syst., Man, Cyben., vol. 22, no. 6, pp. 1414-1427, 1992.
40 M. Sugeno and T. Yasukawa, “A fuzzy-logic-based approach to qualitative modeling,” IEEE Trans. Fuzzy Systems, vol. 1, no. 1, pp. 7-31, 1993.
41 H. Ishibuchi, K. Nozaki, and H. Tanaka, “Distributed representation of fuzzy rules and its application to pattern classification,” Fuuy Sets and syst., vol.52, pp 21-32, 1992.
Mr. E.Kiran Kumar
Dadi Institute of Engineering and Technology - India
Mr. S.R.C.Kishore
Pydah College of Engineering - India
Mr. P.V.V.Kishore
Andhra University - India
pvvkishore@gmail.com
Dr. P.Rajesh Kumar
Andhra University - India