Captioning on Glass

Google Glass application utilizing real-time captioning for the deaf and hard-of-hearing.

Problem

People who are deaf or hard-of-hearing have certain means of communicating in their day-to-day lives. Those who do not use American Sign Language might wear digital hearing aids, have cochlear implants, or rely on captioning. Current captioning technology requires a skilled stenographer to transcribe speech manually.

Captioning on Glass (CoG) is a Google Glass app developed by Jay Zuerndorfer and Thad Starner at Georgia Tech that provides real-time captioning with speech-to-text technology, allowing those who are deaf or hard-of-hearing to converse with others in near real-time.

Our goal was to evaluate and re-design the user experience on the existing version of the software to provide a more immersive and seamless experience to users.

User Research and Evaluation

We recruited deaf and hard-of-hearing participants from ADAPTS (Access Disabled Assistance Program for Tech Students) disability service program, AARP (American Association of Retired Persons), HLAA (Hearing Loss Association of America), as well as personal connections.

We targeted two main user groups: the hard-of-hearing and the late deaf. From extensive user research, we have found that members of these groups rely on either hearing aids or a heightened ability to read lips to understand speech. Real-time captioning gives these users more flexibility and eases their one-on-one communication.

Wireframes

How it Works

Users on the Android side speak into the device, which get converted into text onto the Google Glass app.