How Google Glass Can Change How Kids with Autism Read and Speak
Typically, children with autism struggle communicating with others in back and forth conversations. For this reason, researchers at the University of Toronto have developed an app called Holli, which works with Google Glass and serves as an aid for children with ASD.
The software has a program that assists children in matching facial expressions with feelings. As a child is looking at another individual speak, the Google Glass camera records what the child sees and sends data to translate the spoken word into text. From there, a set of responses (happy, sad or mad) are generated to the Google Glass screen, in accordance with the conversation’s theme. Once the child replies back, saying one of the responses, the generated responses disappear from the screen. The app then listens to the next speaker in the conversation – displaying the word ‘listening’during the conversation. Both researchers and parents can watch the videos later on. Overall, the Holli system is 90% accurate during the ‘listening’ process, with an average response time of 2.5 seconds.
Initial research for Holli
The initial study conducted for the Holli system involved the participation of 15 children with ASD, aged 8-16, who were able to read without glasses. “We developed software for a wearable system that helps coach children with autism spectrum disorder in everyday social interactions. In this study, we show that children are able to use this new technology and they enjoy interacting with it” said Azadeh Kushki, an assistant professor at the Institute of Biomaterials and Biomedical Engineering at the University of Toronto.
The children were given a brief overview of Holli and then they were directed through a restaurant-themed conversation, where the research assistant played the role of a restaurant staff member. The entire process involved a total of ten interactions, Next, the children were asked to fill out a questionnaire created by the researchers. The ending results revealed that all the participants successfully completed the process involving the ten interactions while using the Holli system. “The interesting thing about our new technology is that we are not trying to replace human-to-human interactions; instead, we use this app to coach children who are communicating with people in real-world situations. Children can practice their skills outside of their normal therapy sessions and it can provide them with increased independence in everyday interactions” said Professor Kushki.
Very promising results
The researchers wrote in their study that ‘this result demonstrates the speech recognition response time is robust enough to process speech, make appropriate predictions, and generate responses in real time’. “This study shows the potential of technology-based intervention to help children with ASD. These systems can be used in everyday settings, such as home and school, to reinforce techniques learned in therapeutic settings” stated Professor Kushki. Furthermore, researchers expressed that the results will allow for further development of customization tools according to each child’s unique preference, as well as the future development of similar apps. “Technology has tremendous potential to change the way we think about delivering services to those with ASD. It can augment existing face-to-face interventions to make services accessible in a timely and cost-effective way and help increase treatment effectiveness” concluded Professor Kushki.
The Autism Glass Project
Similarly, the Autism Glass Project, introduced by researchers at Stanford University, is also gaining recognition to build at-home treatment plans for autism. The project involves a Google Glass system using machine learning and artificial intelligence to recognize facial expressions and produce social signals. Catalin Voss, founder of the Autism Glass Project, stated “we can literally jump back here and say, 'This was the angry moment,'.You look at me, the camera is tracking my face, and essentially it sends those frames to the phone, which does all the computation, and sends back the social cue.” The project’s second phase study involves the participation of 100 autistic children who:
- Have been diagnosed with ASD
- Are between the ages of 3 and 17
- Receive at home ABA therapy more than two times per week
The second phase of the study will last for several months, during which time,researchers ask that families come in for five face-to-face appointments over the course of three months. Each family will receive an Android phone, a pair of Google Glasses and the study app to use at home, several times per week for around 20 minutes. “You can start to ask questions, of the percentage of time that the child is talking to his mom, how much is the child looking at her?” said Voss. The team will monitor performance and conduct analysis to show how visual engagement can improve emotional recognition over a prolonged period of time. “The autism glass program is meant to teach children with autism how to understand what a face is telling them. And we believe that when that happens, they will become more socially engaged” said Dennis Wall, head investigator of the Wall Lab at the Stanford University School of Medicine.
The study will take place for 6 weeks, after which the families will need to attend a follow-up appointment. As large as the second phase of the project is, further research and work will need to be done in order to allow the team to improve their software and expand their techniques. “If you think about it, what we’re doing is giving children with autism superpowers” said Voss.
Final thoughts
There are several children with autism who are learning to read facial expressions by working with therapists and using flashcards portrayed with different emotions. Yet, nowadays with continuous innovative advancements, technology has become an inspiring learning approach for children with autism. Several studies have found that children’s concentration, communication, and social skills can improve with the use of computers or tablets. In fact, a large percentage of children with ASD have shown a preference for the use of touchscreens, thus supporting the prospect of technology as an effective teaching tool.
The researchers at Standford University hope that the new Autism Glass Project can help provide reasonably priced at-home treatment.“Glass and wearable technology are the future. They’re going to play a pivotal role in how we understand, manage and diagnose disorders like autism” said Robert Ring, chief science officer at Autism Speaks.
References
http://www.autism-society.org/what-is/facts-and-statistics/
https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml
https://www.cbsnews.com/news/google-glass-could-help-children-with-autism/
https://www.disabled-world.com/assistivedevices/apps/asd-glass.php
http://autismglass.stanford.edu
http://nypost.com/2016/06/23/google-glass-could-change-the-way-autistic-kids-read-faces/
https://techcrunch.com/2015/10/19/stanford-researchers-treat-autism-with-google-glass/
https://www.frontiersin.org/articles/10.3389/frobt.2017.00031/full#h1
