Autism and speech

Two decades of work on speech and social communication: from measuring emotion and prosody in speech, to games and virtual humans that help autistic children and teens practice, to clinically aligned Arabic speech assessment today.

A speech therapy game built with children on the autism spectrum at the MIT Media Lab: louder, clearer speech moves the turtle.
2006first work on recognizing emotion from speech (IVA Best Paper nominee)
LISSAvirtual human that helps autistic teens practice conversation
r = 0.79agreement of Harf-Speech with certified speech-language pathologists

2006 to 2008

Speech, prosody and emotion

At the University of Memphis, robust recognition of emotion from speech and the role of prosody in dialogue; then a comparison of speech properties of neurotypical people and people diagnosed with autism or Down syndrome (ACM ASSETS 2008).

2009 to 2010

Speech therapy games with children on the autism spectrum

At the MIT Media Lab, we designed interactive, customizable speech-enabled games with children on the autism spectrum, and studied participatory design with autistic adolescents.

  • Exploring Speech Therapy Games with Children on the Autism Spectrum E. Hoque, J. K. Lane, R. el Kaliouby, M. Goodwin, and R. W. Picard INTERSPEECH 2009, Brighton, 2009 Speech-therapy games designed with children on the autism spectrum.
  • Design and Evaluation of Interactive, Customizable and Extensible Speech Enabled Games as Speech Therapy for Kids with Autism E. Hoque, M. Goodwin, R. el Kaliouby, and R. W. Picard International Meeting for Autism Research (IMFAR 2010), Philadelphia, 2010
  • Lessons from Participatory Design with Adolescents on the Autism Spectrum M. Madsen, R. el Kaliouby, M. Eckhardt, E. Hoque, M. Goodwin, and R. W. Picard ACM CHI 2009, Work-in-Progress, Boston, 2009

2015 to 2020

LISSA: a virtual human for social-skills practice

LISSA (Live Interactive Social Skill Assistance) holds a conversation and gives real-time feedback on social and nonverbal behavior. We tested it with autistic teens, with experimental results and design lessons published at IVA 2016 and IVA 2020.

2025 to 2026

Autism in Saudi Arabia, and Arabic speech assessment

An estimate of autism prevalence among children in Saudi Arabia from multiple data sources, and Harf-Speech, which scores Arabic pronunciation phoneme by phoneme on a clinical scale (Interspeech 2026), a building block for scalable Arabic speech therapy.

← All research