Captioning on Glass
Google Glass application utilizing real-time captioning for the deaf and hard-of-hearing.
People who are deaf or hard-of-hearing have certain means of communicating in their day-to-day lives. Those who do not use American Sign Language might wear digital hearing aids, have cochlear implants, or rely on captioning. Current captioning technology requires a skilled stenographer to transcribe speech manually.
Captioning on Glass (CoG) is a Google Glass app developed by Jay Zuerndorfer and Thad Starner at Georgia Tech that provides real-time captioning with speech-to-text technology, allowing those who are deaf or hard-of-hearing to converse with others in near real-time.
Our goal was to evaluate and re-design the user experience on the existing version of the software to provide a more immersive and seamless experience to users.
- Heuristic evaluation
- User research
- User testing
- Interviews and diary studies
- Logo and graphic design
- Dr. Jim Foley, project advisor
- Jay Zurendorfer, project initiator
- Dr. Thad Starner, project advisor & instructor
- Brighton Vino Jegarajan
- Anand Sainath
- Robert Solomon
- Qian Xie
User Research and Evaluation
We recruited deaf and hard-of-hearing participants from ADAPTS (Access Disabled Assistance Program for Tech Students) disability service program, AARP (American Association of Retired Persons), HLAA (Hearing Loss Association of America), as well as personal connections.
We targeted two main user groups: the hard-of-hearing and the late deaf. From extensive user research, we have found that members of these groups rely on either hearing aids or a heightened ability to read lips to understand speech. Real-time captioning gives these users more flexibility and eases their one-on-one communication.
How it Works
Users on the Android side speak into the device, which get converted into text onto the Google Glass app.
Captioning on Glass was released October 2, 2014 into the Google Glass store.
It was also entered into the Convergence Innovation Competition where it won both Connected Home and People's Choice categories.