Kinect Sign Language Translator expands communication possibilities for the deaf
Baku, November 5 (AZERTAC). Dedicated researchers in China have created the Kinect Sign Language Translator, a prototype system that understands the gestures of sign language and converts them to spoken and written language—and vice versa. The system captures a conversation from both sides: the deaf person is shown signing, with a written and spoken translation being rendered in real-time, while the system takes the hearing person's spoken words and turns them into accurate, understandable signs.
This project was a result of collaboration, facilitated by Microsoft Research Connections, between the Chinese Academy of Sciences, Beijing Union University, and Microsoft Research Asia, each of which made crucial contributions.
Professor Xilin Chen, deputy director of the Institute of Computing Technology at the Chinese Academy of Sciences, has spent much of the past decade studying sign language recognition, hoping to devise a way to enable signed communication between people with hearing loss and their hearing neighbors. “We knew that information technology, especially computer technology, has grown up very fast. So from my point of view, I thought this is the right time to develop some technology to help [the deaf community]. That` the motivation,” Chen explained.
Motivation met action when Kinect for Xbox came on the scene. Originally developed for gaming, the Kinect`s sensors read a user`s body position and movements and, with the help of a computer, translate them into commands. It thus has tremendous potential for understanding the complex gestures that make up sign language and for translating the signs into spoken or written words and sentences.
The November 2010 release of Kinect stirred tremendous interest in the research community. That interest intensified with the June 2011 release of the Microsoft-supported Kinect for Windows software development kit (SDK), which helped make the technology broadly available for scientific use.
Also essential to this project was the participation of the special education program at Beijing Union University. “One unique contribution of this project is that it is a joint effort between software researchers and the deaf and hard of hearing,” Zhou says. “A group of teachers and students from Beijing Union University joined this project, and this enabled tests of our algorithms to be conducted on real-world data.”
Other news in this section
