Nowadays, the conversational user interface (CUI) is prevalent and powerful. Hearing-impaired people do not have access to this technology. However, Voca serves as a temporary App to provide accessibility for those people.
Product Designer, Prototyper
2017 Sep. - 2017 Dec.
Led the design and prototyping of accessibility of voice user interface from start to finish, including the early research, user flow, interaction design and prototyping.
Hearing-impaired users can type in the command first, then Voca would translate the text into voice message & speak towards the device. Then the device would compute & respond with a voice message. Voca would pick it up & translate it back to text for the users to see.
I researched how deaf people could use Alexa, Google Assistant, Cortana and Siri to communicate with the virtual assistants. Here are some insights:
We used Echo as an example of a voice-controlled device, and experimented different kinds of errors. Then we concluded with the following flow:
A good communication model to balance good experience & 3 different agents
A practical message translating model which provides feedback & error recovery
A clean visual hierarchy to unify & stratify the same component with different information
An efficient volume increasing model to help the user replay the command if confronting an error
An engaging error recovery flow to guide & educate users how to recover from errors
From the test result, we finalized with our 4th design: Voca acts as neutral textbox, VA (Virtual Assistant) acts as respondent. We chose it because it can deliver a clear communication model to users, and it can also increase exposure of Voca for branding.
After observing the communication process of Echo & recorded message, we refined the interaction process of Echo. Then we iterated on our design by checking the timing of light feedbacks & responses.
After researching the current design solutions & the average number of smart-recommended commands, we realized that there were several problems with design 1.0, such as the same visual design for different information hierarchies, and that the vertical design limits the number of smart-recommended commands.
After testing 2 different replay volume increment solutions, it was found out that it took 3 times to let the device hear the voice message in incremental volume design. However, it only took 1 time for the device to hear in maximal volume design. So we finalized the maximal volume design.
To engage & educate users, we should concise the tips in 1 page & ensure users follow the instructions. So we used the checkbox design & consolidated the volume increasing & distance shortening tips in one page.
In our research, users can switch the virtual assistant in the same device(Cortana in Amazon Echo). Also, it is possible that users would talk to different virtual assistants in different scenarios, so it is necessary to implant this function. In the testing, I put the changing va function into 3 different positions: hamburger menu, VA's legend & virtual assistant icon. The result showed that most testers chose the hamburger menu.
In general, Voca is emphasized on onboarding: getting users aware of the system and user-friendly features, communication model: elevating the brand value and users' understanding of the mechanism, and error recovery: helping users recover from the error-prone scenarios.
When the VA sends back the voice message, Voca would translate the voice message into text with recommended commands for further interacting with the VA.
While waiting for the VA's response, user can click the tip button to check if the device is listening or replay the command.
The tutorial tips would guide users to check the volume is at maximum & the distance is close enough for the replay of the command to be effective.
The user can open the command panel & click the favorited command. Voca would speak the favorited command to the device.
When the user is typing in a command, Voca would smart-recommend related commands for users to choose.
When users need to interact with different virtual assistants (devices), they can change the default virtual assistant and trigger word.
We developed the workable prototype to test the core function of Voca: translating texts into voice messages, and translating voice messages back to texts. Here is the Github link.
We tested the workable prototype by typing in different category of commands, and recording all the texts & translated texts from voice messages. Then we compared the texts with translated texts from voice messages to calculate the accuracy. The following is the chart of the accuracy of Voca's voice message translation:
For design decision making, the majority is not always right. The testing result, however, is.
For the fundamental model or design, it would be always greater to test it in an earlier stage than in a later stage. The adjusting cost is larger if it is later.
Design thinking can be cooperated, the design implementation (wireframe, ui, prototype), however, will compromise if delegating to different designers.
Better said than done. For communication, a hasty sketch is sometimes better than a beautiful speech.
Deploy the communication model
Deploy the message transferring model
Increase the accuracy of voice message transferring
Deploy the error recovery modules
Deploy the system setting module
Deploy the command favorite & recommendation modules