Publications

160 papers, patents and theses, 2005 to 2026. Students are typically first authors. Also on Google Scholar, and in full in the CV.

Show
Topic
Type

2026

  1. HAL: Inducing Human-likeness in LLMs with Alignment M. Hasan, J. Zhao, and E. Hoque arXiv:2601.02813 · Preprint
    In brief

    Human-likeness is hard to define, measure or train for. HAL makes it measurable and uses it as an alignment target, so language models can become more human-like in conversation directly rather than only through scale.

    Paper (arXiv)PDF
  2. Replicating Human Motivated Reasoning Studies with LLMs N. Pate, A. M. Proma, H. He, J. N. Druckman, D. C. Molden, G. Ghoshal, and E. Hoque arXiv:2601.16130 · Preprint
    In brief

    People often reason toward the conclusion they want. We replicated classic motivated-reasoning experiments with base language models to test whether they show the same bias, which matters for using LLMs to simulate survey respondents.

    Paper (arXiv)PDF
  3. Can LLMs Emulate Human Belief Dynamics? A. M. Proma, N. Pate, J. N. Druckman, G. Ghoshal, H. He, and E. Hoque arXiv:2605.18781 · Preprint
    In brief

    We replicated an established study of how beliefs form and change in social networks with 12 language models. The answer is a clear no: the models fail in systematic ways, a caution for using LLMs as stand-ins for people.

    Paper (arXiv)PDF
  4. NeuroSift for Task-Aware Quality Assurance of Multimedia Data in Remote Parkinson Disease Assessment: Machine Learning Model Development and Validation Study M. S. Islam, S. Park, E. X. Ma, T. Adnan, and E. Hoque Journal of Medical Internet Research 28, e91756 · Journal
    In brief

    Videos recorded at home are often too dark, off-frame or incomplete for AI to analyze. NeuroSift checks each recording for the problems that matter for that specific task, so unreliable data can be caught before it skews Parkinson's screening.

    Article (open access)
  5. Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language Therapy A. Azad, M. S. Hossain, M. Shanto, S. Boughorbel, A. Aljouie, B. Alwuqaysi, Y. Bokhari, A. O. Sindi, and E. Hoque arXiv:2607.27212 · Preprint
    In brief

    Autistic children in Arabic-speaking countries face a shortage of speech therapists and almost no culturally grounded digital materials. Digital Harf is a multimodal AI system for Arabic speech and language therapy, built to work alongside clinicians.

    Paper (arXiv)
  6. Scalable AI-Based Clinical Communication Training and Automated Assessment M. Hasan, R. Epstein, T. Carroll, and E. Hoque arXiv:2609.22517 · Preprint
    In brief

    Building on our randomized trial with SOPHIE, this work extends AI-patient practice and automated feedback to a broader range of clinical conversations, so clinicians can practice repeatedly at scale.

    Paper (arXiv)PDF
  7. The Effects of Generative AI Virtual Patient in Serious Illness Communication Skills: Randomized Controlled Trial M. Hasan, K. G. Haut, T. M. Carroll, R. M. Epstein, T. Sen, and E. Hoque JMIR Medical Education · Journal · In press
    In brief

    A randomized controlled trial of SOPHIE, our generative-AI virtual patient for practicing serious-illness conversations. Learners who trained with SOPHIE improved on the core skills these conversations demand: empathizing, being explicit and empowering the patient.

    PDFDOI
  8. Validation of Remote Multimodal AI Screening for Parkinson Disease Across Diverse Settings M. S. Islam, T. Adnan, A. Abdelkader, Z. Liu, E. Ma, S. Park, A. Azad, P. Liu, M. Pawlik, E. Hartman, E. Shelton, K. B. Larson, M. S. Rahman, C. Schwartz, K. Jaffe, J. L. Adams, R. B. Schneider, J. Freyberg, E. R. Dorsey, and E. Hoque Communications Medicine · Journal · In press
    In brief

    Can a few minutes in front of a webcam screen for Parkinson's? This study validates our multimodal PARK screening (speech, facial expression, finger tapping) across diverse settings, a step from lab prototype toward real-world use.

    PDF
  9. AI Can Enhance Creativity in Social Networks R. A. Baten, A. S. Bangash, K. Veera, G. Ghoshal, and E. Hoque npj Artificial Intelligence · Journal · In press
    In brief

    We built SocialMuse, an AI that recommends whom to draw inspiration from in an online idea-generation network. Networks guided by the AI outperformed AI-agnostic networks on several creativity measures and grew more decentralized.

    PDF
  10. PULSAR: Graph-Based Positive Unlabeled Learning with Multi-Stream Adaptive Convolutions for Parkinson's Disease Recognition M. Z. U. Alam, A. Azad, M. S. Islam, E. Hoque, and M. S. Rahman ACM Transactions on Computing for Healthcare 7(2) · Journal
    In brief

    PULSAR screens for Parkinson's from webcam videos of a finger-tapping task, using graph neural networks over hand keypoints. It is aimed at places with little access to neurologists, trained on 382 participants.

    PDF (preprint)DOI
  11. AI Virtual Standardized Patient Training for Perinatal Dental Communication: A Pilot Study S. Cui, M. Hasan, C. Yeo, P. Daga, E. Hoque, N. A. Jallad, and J. Xiao Journal of Dental Education · Journal
    In brief

    A pilot of AI virtual standardized patients for dental students learning to counsel pregnant patients. It extends our clinical-communication training approach beyond medicine.

    DOI
  12. Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment A. Azad, M. S. H. Shanto, M. S. Hossain, B. Alwuqaysi, S. Boughorbel, Y. Bokhari, A. Aljouie, A. O. Sindi, and E. Hoque INTERSPEECH 2026, Sydney · Conference
    In brief

    Validated tools for scoring Arabic pronunciation barely exist. Harf-Speech scores pronunciation phoneme by phoneme on a clinical scale and agrees with certified speech-language pathologists (Pearson r = 0.79), a building block for scalable Arabic speech therapy.

    PDF
  13. Design and Evaluation of a Culturally Adapted Multimodal Virtual Agent for PTSD Screening C. Ozel, W. Nadeem, S. Potter, Y. Bokhari, B. Alwuqaysi, W. Alotaibi, R. F. Alnufaie, S. Boughorbel, A. Aljouie, R. Altasan, and E. Hoque ACM International Conference on Intelligent Virtual Agents (IVA 2026) · Conference
    In brief

    PTSD is widely underreported among military personnel. Molhim is a culturally adapted, LLM-driven virtual avatar that conducts structured screening conversations and administers the PCL-5 checklist in a military healthcare setting.

    PDF (preprint)
  14. Multimodal Remote Digital Phenotyping for Detecting and Tracking Early Parkinsonian Change in LRRK2 Carriers T. M. T. Adnan, A. Abdelkader, M. S. Islam, M. Pawlik, E. Hartman, E. Hoque, and R. B. Schneider Research Square (preprint) · Journal · Preprint
    In brief

    People who carry LRRK2 gene variants are at high risk of Parkinson's. From remote webcam tasks by 829 participants, including 158 LRRK2 carriers, our model told carriers without symptoms apart from controls with 92.9% accuracy, and a continuous score tracked decline in two carriers who later developed Parkinson's.

    PDFPMCDOI
  15. Benchmarking Video Foundation Models for Remote Parkinson's Disease Screening M. S. Islam, E. Hossain, A. Abdelkader, T. Adnan, F. R. Mashrur, S. Park, P. Kumar, Q. Sudais, N. Chunga, N. Shah, J. Freyberg, C. Kanan, R. Schneider, and E. Hoque MICCAI 2026, Strasbourg · Conference
    In brief

    The largest study yet of video foundation models for remote Parkinson's screening: 1,888 participants, 32,847 videos, 16 clinical tasks, seven models. Which model works best depends strongly on the task, which matters for anyone building screening tools.

    PDF (preprint)arXiv
  16. OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization K. Ong, S. Boughorbel, L. Xiao, C. Ekbote, W. Dai, A. Qu, J. Wu, R. Mao, E. Hoque, E. Cambria, G. Mengaldo, and P. P. Liang International Conference on Machine Learning (ICML 2026), Seoul · Conference
    In brief

    A 7B-parameter foundation model for understanding human social behavior across many tasks and modalities, trained with a new reinforcement-learning method that balances uneven behavioral data.

    PDF

2025

  1. AI Standardized Patient Improves Human Conversations in Advanced Cancer Care K. Haut, M. Hasan, T. Carroll, R. Epstein, T. Sen, and E. Hoque arXiv:2505.02694 · Preprint
    In brief

    Practicing with SOPHIE, an AI standardized patient, improved how clinicians handled serious-illness conversations in advanced cancer care, offering a cheaper and more scalable alternative to actors.

    Paper (arXiv)PDF
  2. VisualActBench: Can VLMs See and Act like a Human? D. Zhang, P. Liu, X. Zhou, Y. Ge, G. Lan, J. Bi, C. Brinton, E. Hoque, and J. Luo arXiv:2512.09907 · Preprint
    In brief

    A benchmark that tests whether vision-language models can work out what to do from a scene alone, without being told in words, the way people do.

    Paper (arXiv)PDF
  3. How LLMs Fail to Support Fact-Checking A. M. Proma, N. Pate, J. Druckman, G. Ghoshal, H. He, and E. Hoque arXiv:2503.01902 · Preprint
    In brief

    We tested ChatGPT, Gemini and Claude at countering political misinformation and found consistent gaps, especially in grounding their answers in reliable sources.

    Paper (arXiv)PDF
  4. Personalized Large Language Models Can Increase the Belief Accuracy of Social Networks A. M. Proma, N. Pate, S. Kelty, G. Ghoshal, J. N. Druckman, and E. Hoque arXiv:2506.06153 · Preprint
    In brief

    In networked experiments, personalized LLM responses made not only individuals but whole social networks hold more accurate beliefs on contested issues.

    Paper (arXiv)PDF
  5. New Developments in Artificial Intelligence to Enhance Communication Research and Skills Training E. Hoque, D. Schillinger, Z. Imel, K. Saab, and R. Epstein Patient Education and Counseling 137, 109138 · Workshop, abstract or demo
    In brief

    How AI can move healthcare communication research beyond patient surveys and slow human coding of recorded visits, and how it can help clinicians practice and improve.

  6. Designing Nudges, Not Shortcuts, in the Age of AI E. Hoque and C. Pinhanez IEEE Pervasive Computing 24(2) · Journal
    In brief

    An essay on AI in education: rather than tools that hand students the answers, pervasive computing should design AI that nudges people to think and learn for themselves.

    Article (IEEE)
  7. Exploring the Role of Randomization on Belief Rigidity in Online Social Networks A. M. Proma, N. Pate, R. A. Baten, S. Chen, J. N. Druckman, G. Ghoshal, and E. Hoque IEEE Transactions on Affective Computing · Journal
    In brief

    Social media can harden beliefs through echo chambers. Online experiments test whether adding randomness to who people are connected with loosens belief rigidity, a simple network-level intervention.

    PDF (preprint)
    In the press University of Rochester
  8. 'Poker with Play Money': Exploring Psychotherapist Training with Virtual Patients C. M. Baseman, M. Hasan, N. Swinger, S. A. M. Rauch, E. Hoque, and R. I. Arriaga Proceedings of the ACM on Human-Computer Interaction 9(7), CSCW · Journal
    In brief

    What happens when psychotherapists train with virtual patients? This CSCW study examines how practicing with AI patients fits into therapist training.

    PDFDOI
  9. Estimating Autism Prevalence Among Children in the Kingdom of Saudi Arabia: A Comprehensive Multisource Benchmarking Approach H. Khamees, A. Gulati, A. Vatsa, M. Aldosari, and E. Hoque Saudi Medical Journal 46(11) · Journal
    In brief

    Combines multiple data sources to estimate how common autism is among children in Saudi Arabia, filling a gap in national planning data for diagnosis and services.

    PDF
  10. AI-Enabled Parkinson's Disease Screening Using Smile Videos T. Adnan, M. S. Islam, S. Lee, E. M. W. R. Chowdhury, S. D. Tithi, K. Noshin, M. R. Islam, I. Sarker, M. S. Rahman, R. B. Schneider, J. L. Adams, E. R. Dorsey, and E. Hoque NEJM AI 2(7) · Journal
    In brief

    A smile recorded on a webcam can flag Parkinson's. Using facial-expression videos from 1,059 people recorded at home and in clinics, the model detects reduced facial expressivity (hypomimia) and was tested on external videos it never saw in training. Published in NEJM AI.

    Preprint (arXiv)DOI
  11. A Novel Fusion Architecture for PD Detection Using Semi-Supervised Speech Embeddings T. Adnan, A. Abdelkader, Z. Liu, E. Hossain, S. Park, M. S. Islam, and E. Hoque npj Parkinson's Disease 11(1) · Journal
    In brief

    Screening for Parkinson's from one spoken sentence recorded at home. A fusion of modern speech embeddings, trained on 1,306 people (392 with PD), beats traditional acoustic features and holds up across demographics.

    PDFarXiv
  12. Don't Follow the Leader: Independent Thinkers Create Scientific Innovation S. Kelty, R. A. Baten, A. M. Proma, E. Hoque, J. Bollen, and G. Ghoshal Humanities and Social Sciences Communications · Journal
    In brief

    In science, following the most visible leaders tends to produce less novel work. Independent thinkers are more likely to create scientific innovation.

  13. Hi5: Synthetic Data for Inclusive, Robust, Hand Pose Estimation M. Hasan, C. Ozel, N. Long, A. Martin, S. Potter, T. Adnan, S. Lee, A. Zadeh, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2025) · Conference
    In brief

    Hand-pose models underrepresent many skin tones and natural gestures. Hi5 is a synthetic dataset of 583,000 diverse, annotated hand images; models trained only on it match those trained on human-labeled data and are more robust.

    PDF (preprint)
  14. Beyond Accuracy: Enhancing Parkinson's Diagnosis with Uncertainty Quantification of Machine Learning Models A. Azad, M. S. Islam, E. Hoque, and M. S. Rahman International Conference on AI in Healthcare (AIiH 2025), University of Cambridge · Conference · Best Paper Nomination
    In brief

    A Parkinson's prediction is only useful if you know when to trust it. This work adds uncertainty quantification to PD screening models so low-confidence cases can be flagged for clinicians. Best Paper nominee.

    Full text (PMC)
  15. Personalizing LLM Responses to Combat Political Misinformation A. M. Proma, N. Pate, J. N. Druckman, G. Ghoshal, and E. Hoque ACM Conference on User Modeling, Adaptation and Personalization (UMAP 2025) · Conference
    In brief

    Can a chatbot correct political misinformation better if it tailors its answer to the reader? This study tests personalized LLM responses as a counter to misinformation.

    PDF
  16. Accessible, At-Home Detection of Parkinson's Disease via Multi-task Video Analysis M. S. Islam, T. Adnan, J. Freyberg, S. Lee, A. Abdelkader, M. Pawlik, C. Schwartz, K. Jaffe, R. B. Schneider, E. R. Dorsey, and E. Hoque AAAI Conference on Artificial Intelligence (AAAI 2025) · Conference
    In brief

    Parkinson's affects movement, face and voice, so we combine all three. UFNet fuses webcam videos of finger tapping, facial expression and speech from 845 people and weighs each task by its uncertainty, improving at-home detection (AAAI 2025).

    PDFarXiv

2024

  1. SOPHIE: Testing a Virtual, Interactive, AI-Augmented End-of-Life Communication Training Tool K. G. Haut, R. Epstein, T. M. Carroll, B. Kane, L. Schubert, and E. Hoque Journal of Pain and Symptom Management 67(5), e794-e795 (abstract) · Workshop, abstract or demo
    In brief

    Conference presentation of SOPHIE as an alternative to traditional communication workshops for end-of-life conversations, and the design of the study testing it.

    Abstract
  2. Managing Emotional Dialogue for a Virtual Cancer Patient: A Schema-Guided Approach B. Kane, C. Giugno, L. Schubert, K. Haut, C. Wohn, and E. Hoque IEEE Transactions on Affective Computing 15(3) · Journal
    In brief

    How do you make a virtual cancer patient respond with realistic emotion? A schema-guided dialogue manager lets SOPHIE handle open-ended, emotionally charged conversations with doctors in training.

    Earlier version (arXiv)
  3. A User-Centered Framework to Empower People with Parkinson's Disease W. Rahman, A. Abdelkader, S. Lee, P. Yang, M. S. Islam, T. Adnan, M. Hasan, E. Wagner, S. Park, E. R. Dorsey, C. Schwartz, K. Jaffe, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 7(4) · Journal
    In brief

    Designed with people with Parkinson's, this framework turns at-home video screening into feedback and resources users can act on.

    PDF
  4. A Survey on Open Information Extraction from Rule-based Model to Large Language Model Y. Zhang, P. Liu, W. Gao, W. Dong, L. Ai, Z. Gong, S. Huang, Z. Li, E. Hoque, and J. Hirschberg Empirical Methods in Natural Language Processing (EMNLP 2024) · Conference
    In brief

    A survey of open information extraction, from rule-based systems to large language models.

    PDF
  5. Getting on the Right Foot: Using Observational and Quantitative Methods to Evaluate Movement Disorders J. Spann, S. Chen, T. Ashizawa, and E. Hoque ACM Conference on Intelligent User Interfaces (IUI 2024) · Conference
    In brief

    Combines clinical observation and quantitative video analysis to evaluate gait in movement disorders.

    PDF

2023

  1. TextMI: Textualize Multimodal Information for Integrating Non-Verbal Cues in Pre-Trained Language Models M. K. Hasan, M. S. Islam, S. Lee, W. Rahman, I. Naim, M. I. Khan, and E. Hoque arXiv:2303.15430 · Preprint
    In brief

    Instead of changing a language model's architecture, TextMI describes facial expressions and tone of voice in words, so an ordinary pretrained language model can use them to understand sentiment and humor.

    Paper (arXiv)PDF
  2. Using AI to Measure Parkinson's Disease Severity at Home M. S. Islam, W. Rahman, A. Abdelkader, S. Lee, P. T. Yang, J. L. Purks, J. L. Adams, R. B. Schneider, E. R. Dorsey, and E. Hoque npj Digital Medicine 6 · Journal
    In brief

    Tap your fingers in front of a webcam and AI rates Parkinson's severity. On 250 people worldwide, the model outperformed a certified rater (error 0.59 vs 0.79) and approached expert neurologists.

    PDFArticle
    In the press University of Rochester
  3. SEER: Sustainable E-commerce with Environmental-impact Rating M. S. Islam, A. M. Proma, C. Wohn, K. Berger, S. Uong, V. Kumar, K. S. Korfmacher, and E. Hoque Cleaner Environmental Systems 8 · Journal
    In brief

    Shoppers say they want sustainable products but rarely buy them. SEER adds explainable environmental-impact ratings to online stores; in a 98-person experiment, users with SEER made more sustainable choices.

    PDF (preprint)Article
  4. Auto-Gait: Automatic Ataxia Risk Assessment with Computer Vision from Gait Task Videos W. Rahman, M. Hasan, M. S. Islam, T. Olubajo, J. Thaker, A. Abdelkader, P. Yang, H. Paulson, G. Oz, A. Durr, T. Klockgether, T. Ashizawa, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    Webcam gait videos analyzed with computer vision to estimate risk of spinocerebellar ataxia, a step toward remote screening for rare movement disorders.

    PDF
  5. Validating a Virtual Human and Automated Feedback System for Training Doctor-Patient Communication Skills K. Haut, C. Wohn, B. Kane, T. Carroll, C. Giugno, V. Kumar, R. Epstein, L. Schubert, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2023), Boston · Conference
    In brief

    SOPHIE, a virtual patient with automated feedback, validated with 30 participants: those who practiced with it outperformed controls in overall communication, empathy and empowering the patient.

    PDF
  6. Unmasking Parkinson's Disease Using Smiles: An AI-Enabled Screening Framework T. Adnan et al. Future of Parkinson's Disease Conference, Austin · Workshop, abstract or demo · Best Abstract in Clinical Research PDF
  7. SAPIEN: Affective Virtual Agents Powered by Large Language Models M. Hasan, C. Ozel, S. Potter, and E. Hoque ACII Workshops and Demos (ACIIW 2023) · Workshop, abstract or demo PDF (preprint)

2022

  1. Novel Idea Generation in Social Networks Is Optimized by Exposure to a 'Goldilocks' Level of Idea-Variability R. A. Baten, R. N. Aslin, G. Ghoshal, and E. Hoque PNAS Nexus 1(5) · Journal
    In brief

    Exposure to too little or too much variety in others' ideas both hurt creativity. A 'Goldilocks' level of idea variability in your social network produces the most novel ideas.

    PDFArticle
  2. A Mental Trespass? Unveiling Truth, Exposing Thoughts and Threatening Civil Liberties with Non-Invasive AI Lie Detection T. K. Sen, K. Haut, D. Lomakin, and E. Hoque IEEE Transactions on Technology and Society 3(3) · Journal
    In brief

    AI lie detection from face, body and voice is coming fast. We argue it threatens civil liberties, and a survey of 129 people shows consent and accuracy drive whether the public accepts it.

    PDFArticle
  3. Discourse Behavior of Older Adults Interacting with a Dialogue Agent Competent in Multiple Topics S. Z. Razavi, L. Schubert, K. Van Orden, M. R. Ali, B. Kane, and E. Hoque ACM Transactions on Interactive Intelligent Systems (TiiS) 12(2) · Journal
    In brief

    How older adults talk with a conversational agent that can discuss many topics, informing companion technology for healthy aging.

    PDF (preprint)Article
  4. KnowUREnvironment: An Automated Knowledge Graph for Climate Change and Environmental Issues M. S. Islam, A. M. Proma, Y. Zhou, S. N. Akter, C. Wohn, and E. Hoque AAAI Fall Symposium: The Role of AI in Responding to Climate Change · Conference
    In brief

    An automated knowledge graph that organizes research and information on climate change and environmental issues.

    PDF
  5. NADBenchmarks: A Compilation of Benchmark Datasets for Machine Learning Tasks Related to Natural Disasters A. M. Proma, M. S. Islam, S. Ciko, R. A. Baten, and E. Hoque AAAI Fall Symposium: The Role of AI in Responding to Climate Change · Conference
    In brief

    A curated catalog of benchmark datasets for machine learning on natural disasters, organized by the disaster-management cycle.

    PDF
  6. Assistive Video Filters for People with Parkinson's Disease to Remove Tremors and Adjust Voice K. Haut, A. Blumenthal, S. Atterbury, X. Zhou, W. Rahman, E. Natali, M. R. Ali, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2022) · Conference · Best Paper Nomination
    In brief

    Real-time video filters that smooth visible tremors and adjust voice, so people with Parkinson's can feel more comfortable on video calls. Best Paper nominee led by undergraduates.

    PDF
  7. Systematic Feature Isolation for Bias Research Using Deepfakes K. Haut, C. Wohn, V. Antony, A. Goldfarb, M. Welsh, D. Sumanthiran, M. R. Ali, and E. Hoque ACM International Conference on Multimedia (ACM MM 2022), Brave New Ideas · Conference · Best Paper Nomination
    In brief

    Uses deepfakes to change one feature of a person at a time, such as appearance, so researchers can isolate the sources of bias in how people are judged. Best Paper nominee.

    PDF
  8. Suggestive Motivational Interviewing Agent for Remote Group Discussion S. Samrose and E. Hoque ACM International Conference on Supporting Group Work (GROUP 2022) · Conference
    In brief

    An AI agent that nudges remote group discussions toward more balanced, motivating conversation.

    PDF
  9. Auto-Gait: Automatic Ataxia Risk Assessment with Computer Vision on Gait Task Videos M. Hasan et al. International Congress for Ataxia Research (ICAR 2022) · Workshop, abstract or demo
    In brief

    Webcam gait videos analyzed with computer vision to estimate risk of spinocerebellar ataxia, a step toward remote screening for rare movement disorders.

    PDF
  10. A Week in the Life with Parkinson's Disease: A Holistic Overview from Four Digital Technologies E. A. Hartman et al. International Congress of Parkinson's Disease and Movement Disorders (MDS 2022), Madrid · Workshop, abstract or demo
  11. Analyzing Gait Videos to Identify and Evaluate Spinocerebellar Ataxia Types 1 and 3 P. T. Yang et al. International Congress of Parkinson's Disease and Movement Disorders (MDS 2022), Madrid · Workshop, abstract or demo
  12. Longitudinal Evaluation of Hypomimia in Individuals with Parkinson's Disease Using a Video Analytics Tool P. T. Yang et al. International Congress of Parkinson's Disease and Movement Disorders (MDS 2022), Madrid · Workshop, abstract or demo

2021

  1. Could You Become More Credible by Being White? Assessing Impact of Race on Credibility with Deepfakes K. Haut, C. Wohn, V. Antony, A. Goldfarb, M. Welsh, D. Sumanthiran, J.-z. Jang, M. R. Ali, and E. Hoque arXiv:2102.08054 · Preprint
    In brief

    We used deepfakes to change a speaker's apparent race on video while keeping everything else the same, to test how race alone changes how credible people seem in video calls.

    Paper (arXiv)PDF
  2. Exploring the Use of Real-Time Camera Filters on Embodiment and Creativity J. Leong, P. Pataranutaporn, Y. Mao, F. Perteneder, E. Hoque, J. M. Baker, and P. Maes CHI 2021 Extended Abstracts · Workshop, abstract or demo
    In brief

    Real-time camera filters change how we look on video calls. This study with the MIT Media Lab explored whether they also change how people feel and think, including how creative they are.

    Paper (ACM)
  3. Detecting Parkinson's Disease from an Online Speech-Task: Observational Study W. Chowdhury, S. Lee, M. S. Islam, V. Antony, H. Ratnu, M. R. Ali, A. Mamun, E. Wagner, S. Jensen-Roberts, E. Waddell, T. Myers, M. Pawlik, J. Soto, M. Coffey, A. Sarkar, R. Schneider, C. Tarolli, K. Lizarraga, J. Adams, M. Little, E. R. Dorsey, and E. Hoque Journal of Medical Internet Research 23(10) · Journal
    In brief

    Anyone, anywhere can record one sentence online to screen for Parkinson's. With 726 participants across the US and beyond, speech features alone reached 0.75 AUC.

    PDFArticle
  4. DBATES: Distinguishing Benefits of Audio, Textual, and Facial Expression Features in Competitive Debate Speeches T. K. Sen, G. Naven, L. Gerstner, D. Bagley, R. A. Baten, W. Chowdhury, M. K. Hasan, K. Haut, A. Mamun, S. Samrose, A. S. Slowe, E. Barnes, M. Frank, and E. Hoque IEEE Transactions on Affective Computing · Journal
    In brief

    A dataset and analysis of competitive debate speeches showing how much audio, text and facial expression each contribute to judging a speaker.

    PDF
  5. Cues to Gender and Racial Identity Reduce Creativity in Diverse Social Networks R. A. Baten, R. N. Aslin, G. Ghoshal, and E. Hoque Scientific Reports 11, 10261 · Journal
    In brief

    In diverse online groups, cues to participants' gender and racial identity reduced creativity, a finding with direct implications for how collaborative platforms present people.

    PDFArticle
    In the press University of Rochester
  6. Analyzing Head Pose in Remotely-Collected Videos of People with Parkinson's Disease M. R. Ali, T. K. Sen, Q. Li, R. Langevin, T. Myers, S. Sharma, E. R. Dorsey, and E. Hoque ACM Transactions on Computing for Healthcare 2(4) · Journal
    In brief

    Head movement in home-recorded videos carries signal about Parkinson's, supporting remote assessment without special hardware.

    PDF
  7. Video-Based Analyses of Parkinson's Disease Severity: A Review K. Sibley, C. Girges, E. Hoque, and T. Foltynie Journal of Parkinson's Disease · Journal
    In brief

    A review of how video analysis is being used to measure Parkinson's severity, and what is needed for clinical use.

    Full text (PMC)Article
  8. A Patient-Centered Digital Scribe for Automatic Medical Documentation J. Wang, M. Lavender, P. Brophy, E. Hoque, and H. Kautz JAMIA Open 4(1) · Journal
    In brief

    A digital scribe that drafts medical documentation from doctor-patient conversations, so clinicians can look at patients instead of screens.

    PDFArticle
  9. Novel Computational Linguistic Measures, Dialogue System and the Development of SOPHIE: Standardized Online Patient for Healthcare Interaction Education M. R. Ali, T. K. Sen, B. Kane, S. Bose, T. Carroll, R. Epstein, L. Schubert, and E. Hoque IEEE Transactions on Affective Computing · Journal
    In brief

    Introduces SOPHIE, a virtual standardized patient for practicing end-of-life conversations, with computational measures of how well clinicians communicate.

    PDF
    In the press HealioUniversity of RochesterThe National Academies
  10. Hitting Your MARQ: Multimodal ARgument Quality Assessment in Long Debate Video M. K. Hasan, J. Spann, M. Hasan, M. S. Islam, K. Haut, R. Mihalcea, and E. Hoque Empirical Methods in Natural Language Processing (EMNLP 2021) · Conference
    In brief

    Assessing the quality of arguments in long debate videos by combining what speakers say with how they say it.

    PDF
  11. Humor Knowledge Enriched Transformer for Understanding Multimodal Humor M. K. Hasan, S. Lee, W. Rahman, A. Zadeh, R. Mihalcea, L.-P. Morency, and E. Hoque AAAI Conference on Artificial Intelligence (AAAI 2021) · Conference
    In brief

    Teaching AI to get jokes: a transformer that combines words, voice and facial cues with external humor knowledge to recognize humor in video.

    PDF
  12. HirePreter: A Framework for Providing Fine-grained Interpretation for Automated Job Interview Analysis W. Rahman, S. Mahbub, A. Salekin, M. K. Hasan, and E. Hoque Workshop on Applied Multimodal Affect Recognition (AMAR), ACII 2021 · Workshop, abstract or demo PDF
  13. Quantifying the Intensity of Toxicity for Discussions and Speakers S. Samrose and E. Hoque Workshop on Applied Multimodal Affect Recognition (AMAR), ACII 2021 · Workshop, abstract or demo PDF
  14. Individual-level Anxiety Detection and Prediction from Longitudinal YouTube and Google Search Engagement Logs A. Zaman, B. Zhang, V. Silenzio, E. Hoque, and H. Kautz Data for the Wellbeing of Most Vulnerable Workshop, ICWSM 2021 · Workshop, abstract or demo PDF
  15. Removing Racial Bias in TED Talk Ratings by Awareness of Verbal and Gesture Quality A. Chattoraj, R. Acharyya, S. Das, M. I. Tanveer, and E. Hoque Responsible AI Workshop, ICLR 2021 · Workshop, abstract or demo PDF

2020

  1. Immediate or Reflective? Effects of Real-Time Feedback on Group Discussions over Videochat S. Samrose, R. Rawassizadeh, and E. Hoque arXiv:2011.06529 · Preprint
    In brief

    A video-chat system that analyzes a group discussion automatically and gives feedback either during the conversation or afterward. We compared how each kind of feedback changes discussions between people who disagree.

    Paper (arXiv)PDF
  2. Upskilling the Future Workforce Using AI and Affective Computing E. Hoque ICMI 2020 Companion (keynote) · Workshop, abstract or demo
    In brief

    A keynote on using AI not to replace people but to amplify human abilities, especially the social and communication skills that set us apart.

    Abstract (ACM)
  3. Aging and Engaging: A Pilot Randomized Controlled Trial of an Online Conversational Skills Coach for Older Adults M. R. Ali, E. Hoque, P. Duberstein, L. Schubert, S. Z. Razavi, B. Kane, C. Silva, J. S. Daks, M. Huang, and K. Van Orden American Journal of Geriatric Psychiatry · Journal
    In brief

    A pilot randomized trial of an online conversation coach that helps older adults practice social engagement, aimed at reducing isolation.

    PDF
  4. Technology-Driven Alteration of Nonverbal Cues and Its Effects on Negotiation R. A. Baten and E. Hoque Negotiation Journal · Journal
    In brief

    Video tools and deepfakes can now change how we look and sound in real time. We examine what that means for negotiation, and the ethical questions it raises.

    PDF (preprint)arXiv
  5. The Relationship Between Deteriorating Mental Health Conditions and Longitudinal Behavioral Changes in Google and YouTube Usage Among College Students in the United States During COVID-19: Observational Study A. Zaman, B. Zhang, V. Silenzio, H. Kautz, and E. Hoque JMIR Mental Health · Journal
    In brief

    During COVID-19, changes in college students' Google and YouTube use tracked worsening mental health, pointing to passive, privacy-aware early warning signs.

    Full text (PMC)Article
  6. Creativity in Temporal Social Networks: How Divergent Thinking Is Impacted by One's Choice of Peers R. A. Baten, D. Bagley, A. Tenesaca, F. Clark, J. P. Bagrow, G. Ghoshal, and E. Hoque Journal of the Royal Society Interface · Journal
    In brief

    Who you choose to draw ideas from shapes how creative you become. Online experiments trace how peer choices affect divergent thinking over time.

    PDF
  7. Leveraging Free-Hand Sketches for Potential Screening of PTSD F. Shahid, W. Rahman, M. S. Rahman, S. Purabi, A. Seddiqa, M. Mostakim, F. Feroz, T. R. Soron, F. Hossain, N. Khan, A. B. Islam, N. Paul, E. Hoque, and A. Islam Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    Simple free-hand drawings may help screen for PTSD, offering a low-cost tool where clinicians are scarce.

    PDF
  8. Are You Really Looking at Me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video M. Tran, T. K. Sen, K. G. Haut, M. R. Ali, and E. Hoque IEEE Transactions on Affective Computing · Journal
    In brief

    Estimates whether people are making eye contact using ordinary video, with no eye tracker needed.

    PDF
  9. Deep Phenotyping of Parkinson's Disease E. R. Dorsey et al. Journal of Parkinson's Disease · Journal
    In brief

    A multi-site effort to characterize Parkinson's in depth with digital and clinical measures.

    Full text (PMC)Article
  10. Using Artificial Intelligence to Analyze and Teach Communication in Healthcare P. Butow and E. Hoque The Breast 50 · Journal
    In brief

    How AI can analyze and help teach communication between clinicians and patients.

    PDFArticle
  11. Discovering Intimate Partner Violence from Web Search History A. Zaman, H. Kautz, E. Hoque, V. Silenzio, C. Nichols, and C. Cerulli IEEE/ACM Conference on Connected Health (CHASE 2020) · Conference
    In brief

    Web search histories can reveal signs of intimate partner violence, suggesting new ways to reach people who may not seek help.

    PDF
  12. A Virtual Conversational Agent for Teens with Autism Spectrum Disorder: Experimental Results and Design Lessons M. R. Ali, S. Z. Razavi, R. Langevin, A. Al-Mamun, B. Kane, R. Rawassizadeh, L. Schubert, and E. Hoque ACM Intelligent Virtual Agents (IVA 2020), Glasgow · Conference
    In brief

    A virtual agent that lets autistic teens practice conversation and get feedback, with experimental results and design lessons.

    PDF
  13. Integrating Multimodal Information in Large Pretrained Transformers W. Rahman, M. K. Hasan, S. Lee, A. Zadeh, C. Mao, L.-P. Morency, and E. Hoque Association for Computational Linguistics (ACL 2020) · Conference
    In brief

    Lets BERT-style language models take in voice and facial cues, not only words, improving sentiment and behavior understanding.

    PDF
  14. Spatio-Temporal Attention and Magnification for Classification of Parkinson's Disease from Videos Collected via the Internet M. R. Ali, J. Hernandez, E. R. Dorsey, E. Hoque, and D. McDuff IEEE Automatic Face and Gesture Recognition (FG 2020) · Conference
    In brief

    Magnifying subtle motion in videos collected online helps AI classify Parkinson's.

    PDF
  15. Leveraging Shared and Divergent Facial Expression Behavior Between Genders in Deception Detection G. Naven, T. K. Sen, L. Gerstner, K. G. Haut, M. Wen, and E. Hoque IEEE Automatic Face and Gesture Recognition (FG 2020) · Conference
    In brief

    Men and women show different facial cues when lying. Modeling those differences improves deception detection.

    PDF
  16. Detecting Individuals with Depressive Disorder from Personal Google Search and YouTube History Logs B. Zhang, A. Zaman, R. Acharyya, E. Hoque, V. Silenzio, and H. Kautz Machine Learning in Public Health Workshop, NeurIPS 2020 · Workshop, abstract or demo PDFarXiv

2019

  1. Predicting TED Talk Ratings from Language and Prosody M. I. Tanveer, M. K. Hasan, D. Gildea, and E. Hoque arXiv:1906.03940 · Preprint
    In brief

    Using more than 2,200 TED talks and 5.5 million viewer ratings, we predicted how audiences rate a talk (inspiring, funny, persuasive) from what speakers say and how they say it.

    Paper (arXiv)PDF
  2. A Causality-Guided Prediction of the TED Talk Ratings from the Speech-Transcripts Using Neural Networks M. I. Tanveer, M. K. Hasan, D. Gildea, and E. Hoque arXiv:1905.08392 · Preprint
    In brief

    Neural networks that predict TED talk ratings from transcripts, designed with a causal model so that predictions are not skewed by confounds such as a speaker's popularity.

    Paper (arXiv)PDF
  3. The PARK Framework for Automated Analysis of Parkinson's Disease Characteristics R. Langevin, M. R. Ali, T. Sen, C. Snyder, T. Myers, E. R. Dorsey, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    PARK: a web tool that guides people through standard Parkinson's tasks at home and analyzes the recorded videos, the foundation of our remote screening work.

    PDF
    In the press NSF
  4. UR-FUNNY: A Multimodal Language Dataset for Understanding Humor M. K. Hasan, W. Rahman, A. Zadeh, J. Zhong, M. I. Tanveer, L.-P. Morency, and E. Hoque Empirical Methods in Natural Language Processing (EMNLP 2019), Hong Kong · Conference
    In brief

    UR-FUNNY: a multimodal dataset of words, voice and gestures from TED talks for teaching AI to understand humor.

    PDF
  5. Facial Expression Based Imagination Index and a Transfer Learning Approach to Detect Deception M. K. Hasan, W. Rahman, L. Gerstner, T. K. Sen, S. Lee, K. G. Haut, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK · Conference
    In brief

    Facial expressions reveal when people are imagining rather than recalling, a signal useful for detecting deception.

    PDF
  6. What Computers Can Teach Us About Doctor-Patient Communication: Leveraging Gender Differences in Cancer Care M. R. Ali, T. Sen, V. Nguyen, R. Rawassizadeh, P. Duberstein, R. Epstein, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK · Conference
    In brief

    Computational analysis of cancer-care conversations reveals gender differences in doctor-patient communication.

    PDF
  7. Upskilling Together: How Peer-Interaction Influences Speaking-Skills Development Online R. A. Baten, F. Clark, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK · Conference
    In brief

    Speakers practicing in an online community improved faster when they interacted with peers.

    PDF
  8. Visual Cues for Disrespectful Conversation Analysis S. Samrose, W. Chu, C. He, Y. Gao, S. S. Shahrin, Z. Bai, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK · Conference
    In brief

    Visual cues that signal disrespect in conversations, a step toward tools for healthier discussions.

    PDF
  9. LIWC Into the Eyes: Using Facial Features to Contextualize Linguistic Analysis in Multimodal Communication M. K. Hasan, T. K. Sen, Y. Yang, R. A. Baten, K. G. Glenn, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK · Conference
    In brief

    Combines facial expressions with word-level language analysis to better interpret what people mean.

    PDF

2018

  1. It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process B. Hecht, L. Wilcox, J. P. Bigham, J. Schöning, E. Hoque, J. Ernst, Y. Bisk, L. D. Russis, L. Yarosh, B. Anjum, D. Contractor, and C. Wu ACM Future of Computing Academy (2018); arXiv:2112.09544 · Preprint
    In brief

    A proposal, from the ACM Future of Computing Academy, that peer reviewers require computing papers to discuss the plausible negative impacts of their work, from privacy to employment, rather than assuming research is a net good.

    Paper (arXiv)PDF
  2. Development of an Electromyographically Controlled Virtual Arm for Poststroke Motor Rehabilitation M. Zhang, R. Rawassizadeh, E. Hoque, and A. Busza Neurorehabilitation and Neural Repair 32(12), abstract F54 · Workshop, abstract or demo
    In brief

    A virtual arm, controlled by muscle signals from the patient's own arm, for early rehabilitation after stroke when the arm is too weak to move much.

    Abstract
  3. The Holographic Electromyographic Automated Rehabilitation Technology (HEART) Project M. Zhang, S. Liu, E. Hoque, and A. Busza Neurology 90(15 Suppl.), P5.008 · Workshop, abstract or demo
    In brief

    A mixed-reality version of the muscle-controlled virtual arm, shown as a hologram, for upper-limb rehabilitation in people with moderate to severe weakness after stroke.

    Abstract
  4. CoCo: Collaboration Coach for Understanding Team Dynamics During Video Conferencing S. Samrose, R. Zhao, J. White, V. Li, L. Nova, Y. Lu, M. R. Ali, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    CoCo coaches teams during video calls, showing how participation and dynamics affect group performance.

    PDF
  5. Automated Dyadic Data Recorder (ADDR) Framework and Analysis of Facial Cues in Deceptive Communication T. Sen, M. K. Hasan, Z. Teicher, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    A framework for recording and analyzing honest and deceptive conversations online, revealing the facial cues of lying. Covered by Newsweek and The Times.

    PDF
    In the press NewsweekThe TimesDaily MailUniversity of RochesterWXXI
  6. Buildup of Speaking Skills in an Online Learning Community: A Network-Analytic Exploration R. Shafipour, R. A. Baten, M. K. Hasan, G. Ghoshal, G. Mateos, and E. Hoque Palgrave Communications · Journal
    In brief

    Network analysis of an online public-speaking community shows how skills spread among peers.

    PDFArticle
  7. Automated Analysis and Prediction of Job Interview Performance I. Naim, M. I. Tanveer, D. Gildea, and E. Hoque IEEE Transactions on Affective Computing 9 · Journal
    In brief

    AI that predicts how interviewers will rate a candidate from what they say and how they say it, and explains which behaviors matter.

    PDF
  8. The What, When, and Why of Facial Expressions: An Objective Analysis of Conversational Skills in Speed-Dating Videos M. R. Ali, T. K. Sen, D. Crasta, V.-D. Nguyen, R. Rogge, and E. Hoque IEEE Automatic Face and Gesture Recognition (FG 2018) · Conference
    In brief

    Objective analysis of facial expressions in speed-dating videos reveals what, when and why expressions matter in conversation.

    PDF
  9. Say CHEESE: Common Human Emotional Expression Set Encoder Analysis of Smiles in Honest and Deceptive Communication T. K. Sen, M. K. Hasan, M. Tran, M. Levin, Y. Yang, and E. Hoque IEEE Automatic Face and Gesture Recognition (FG 2018), Xi'an · Conference
    In brief

    Smiles differ subtly when people are lying. A smile-encoding model finds those differences in honest and deceptive conversations.

    PDF
  10. How Emotional Trajectories Affect Audience Perception in Public Speaking M. I. Tanveer, S. Samrose, R. A. Baten, and E. Hoque ACM Conference on Human Factors in Computing Systems (CHI 2018), Montreal · Conference
    In brief

    Analyzing TED talks shows how the emotional arc of a talk shapes how audiences rate it.

    PDF
  11. Aging and Engaging: A Social Conversational Skills Training Program for Older Adults M. R. Ali, K. Van Orden, K. Parkhurst, V. D. Nguyen, S. Liu, P. Duberstein, and E. Hoque ACM Conference on Intelligent User Interfaces (IUI 2018), Tokyo · Conference
    In brief

    A conversation-skills training program for older adults, with an AI coach that gives feedback on social engagement.

    PDF
  12. Understanding Social Interpersonal Interaction via Synchronization Templates of Facial Events R. Li, J. Curhan, and E. Hoque AAAI Conference on Artificial Intelligence (AAAI 2018) · Conference
    In brief

    Shows how synchronized facial expressions between two people predict social outcomes such as negotiation results.

    PDF
  13. Analyzing the Impact of Gender on the Automation of Feedback for Public Speaking A. Singhal, M. R. Ali, R. A. Baten, C. Kurumada, E. Marvin, and E. Hoque International Workshop on Human Behavior Understanding (HBU), FG 2018 · Workshop, abstract or demo

2017

  1. Social Skills Training with Virtual Assistant and Real-Time Feedback M. R. Ali and E. Hoque UbiComp 2017 Adjunct · Workshop, abstract or demo
    In brief

    A virtual assistant that lets people practice conversations privately and gives real-time feedback on nonverbal cues such as eye contact and smiling.

    Paper (ACM)
  2. Automatic Identification of Non-Meaningful Body-Movements and What It Reveals About Humans M. I. Tanveer, R. Zhao, and E. Hoque arXiv:1707.04790 · Preprint
    In brief

    From 84 public-speaking videos we extracted 314 body-movement patterns and learned to tell meaningful gestures from mannerisms such as pacing or shifting weight.

    Paper (arXiv)PDF
  3. Aging and Engaging: The Development of an Automated Tool to Teach Social Engagement Skills K. Van Orden, M. R. Ali, K. Parkhurst, P. Duberstein, and E. Hoque American Journal of Geriatric Psychiatry 25(3) · Journal
    In brief

    Early development of an automated tool that teaches social engagement skills to older adults.

    Article
  4. A Semi-Automated, Collaborative, and Online Training Module for Communication Skills R. Zhao, V. Li, H. Barbosa, G. Ghoshal, and E. Hoque Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) · Journal
    In brief

    Combines automated analysis with crowd feedback to help people practice communication skills online.

    PDF
  5. Automated Video Judgement on a Large-Sized Corpus Collected Online L. Chen, R. Zhao, B. Lehman, C. Leong, G. Feng, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2017) · Conference · Best Paper Nomination
    In brief

    Automated scoring of a large corpus of videos recorded online, with results comparable to human judges. Best Paper nominee.

    PDF
  6. Modeling Doctor-Patient Communication with Affective Text Analysis T. Sen, M. R. Ali, P. Duberstein, R. Epstein, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2017) · Conference
    In brief

    Analyzing the emotional content of doctor-patient conversations to model communication in cancer care.

    PDF
  7. Managing Casual Spoken Dialogue Using Flexible Schemas, Pattern Transduction Trees, and Gist Clauses S. Z. Razavi, L. Schubert, M. R. Ali, and E. Hoque Advances in Cognitive Systems (ACS 2017), Troy, NY · Conference
    In brief

    A dialogue manager that keeps casual conversations on track, used in LISSA and later SOPHIE.

    PDF

2016

  1. Methods and Apparatus for Conversation Coach E. Hoque and R. W. Picard US Patent 9,691,296 B2 · Patent · Basis for later conversation-coaching products
    In brief

    The US patent from my PhD on an automated conversation coach; Microsoft later built on the idea in PowerPoint's Speaker Coach.

    Patent
  2. The LISSA Virtual Human and ASD Teens: An Overview of Initial Experiments S. Z. Razavi, M. R. Ali, T. Smith, L. Schubert, and E. Hoque Intelligent Virtual Agents (IVA 2016), Los Angeles · Conference
    In brief

    Initial experiments with LISSA, a virtual human that helps autistic teens practice conversation.

    PDF
  3. ROC Comment: Automated Descriptive and Subjective Captioning of Behavioral Videos M. R. Ali, F. Ciano, R. Zhao, I. Naim, and E. Hoque ACM UbiComp 2016, Heidelberg · Conference
    In brief

    Automatically generates descriptive and subjective comments on videos of people's behavior.

    PDF
  4. AutoManner: An Automated Interface for Making Public Speakers Aware of Their Mannerisms M. I. Tanveer, R. Zhao, K. Chen, Z. Tiet, and E. Hoque ACM Conference on Intelligent User Interfaces (IUI 2016) · Conference
    In brief

    Shows public speakers their own repetitive mannerisms, automatically found in their videos.

    PDF

2015

  1. Unsupervised Extraction of Human-Interpretable Nonverbal Behavioral Cues in a Public Speaking Scenario M. I. Tanveer, J. Liu, and E. Hoque ACM Multimedia (ACM MM 2015) · Conference
    In brief

    Finds interpretable nonverbal cues in public-speaking videos without labels.

    PDF
  2. ROC Speak: Semi-Automated Personalized Feedback on Nonverbal Behavior from Recorded Videos M. Fung, Y. Jin, R. Zhao, and E. Hoque ACM UbiComp 2015, Osaka · Conference
    In brief

    ROC Speak: record a talk in your browser and get automated plus crowd feedback on your nonverbal behavior. Highlighted by Microsoft Research.

    PDF
    In the press Microsoft Research
  3. Vowel Shapes: An Open-Source, Interactive Tool to Assist Singers with Learning Vowels C. Ryan, K. Ciesinski, and E. Hoque ACM UbiComp 2015, Osaka · Conference
    In brief

    An open-source tool that visualizes vowel sounds to help singers learn them.

    PDF
    In the press University of RochesterWXXI
  4. LISSA: Live Interactive Social Skill Assistance M. R. Ali, D. Crasta, L. Jin, A. Baretto, J. Pachter, R. Rogge, and E. Hoque Affective Computing and Intelligent Interaction (ACII 2015) · Conference · Oral presentation
    In brief

    LISSA, a live virtual assistant that gives real-time feedback on social skills during conversation.

    PDF
  5. Automated Prediction of Job Interview Performance: The Role of What You Say and How You Say It I. Naim, M. I. Tanveer, D. Gildea, and E. Hoque IEEE Automatic Face and Gesture Recognition (FG 2015), Slovenia · Conference · Oral presentation
    In brief

    Predicts job-interview ratings from both what candidates say and how they say it.

    PDF
    In the press SmithsonianSky News
  6. Role of Synchronized Facial Expressions to Predict Negotiation Outcome R. Li, J. Curhan, and E. Hoque IEEE Automatic Face and Gesture Recognition (FG 2015), Slovenia · Conference
    In brief

    Facial synchrony between negotiators predicts negotiation outcomes.

    PDF
  7. Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking M. I. Tanveer, E. Lin, and E. Hoque ACM Conference on Intelligent User Interfaces (IUI 2015), Atlanta · Conference
    In brief

    Rhema gives speakers real-time feedback through Google Glass while they talk, without distracting them.

    PDF
    In the press University of RochesterNew York MagazineEngadgetUniversity of RochesterWXXI

2014

  1. Rich Nonverbal Sensing to Enable New Possibilities in Social Skills Training E. Hoque and R. W. Picard IEEE Computer, special issue on Aware Computing · Journal
    In brief

    How rich sensing of nonverbal behavior opens new ways to train social skills.

    PDF
  2. A Google Glass App to Help the Blind with Small Talk M. I. Tanveer and E. Hoque ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2014), Rochester · Workshop, abstract or demo
  3. M.I.D.A.S. Touch: Magnetic Interactive Device for Alternative Sight through Touch T. K. Sen, M. W. Sinko, A. T. Wilson, and E. Hoque ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2014), Rochester · Workshop, abstract or demo
  4. Processing Nonverbal Behavior in the Cloud M. Fung, J. Curhan, and E. Hoque Academy of Management Annual Meeting (AOM 2014), Philadelphia · Workshop, abstract or demo

2013

  1. Computers to Help with Conversations: Affective Framework to Enhance Human Nonverbal Skills E. Hoque PhD thesis, MIT Media Lab · Thesis
    In brief

    My PhD thesis: an affective computing framework that helps people improve their nonverbal conversation skills.

    PDF
  2. MACH: My Automated Conversation coacH E. Hoque, M. Courgeon, J.-C. Martin, B. Mutlu, and R. W. Picard ACM UbiComp 2013, Zurich · Conference · Best Paper Award (top 5 of 392). UbiComp 10-Year Impact Award, 2023
    In brief

    MACH, a 3D virtual coach that reads your face and voice to help you practice job interviews. UbiComp Best Paper (2013) and 10-Year Impact Award (2023).

    PDFMIT News
    In the press MIT NewsMIT Technology ReviewNew ScientistEngadgetFast CompanyHuffington PostInc.Medical DailyCTVDesign NewsWGBHCJADUniversity of Rochester
  3. Automated Coach to Practice Conversations E. Hoque and R. W. Picard Affective Computing and Intelligent Interaction (ACII 2013) · Workshop, abstract or demo PDF

2012

  1. Mood Meter: Counting Smiles in the Wild J. Hernandez*, E. Hoque*, W. Drevo, and R. W. Picard (*equal contribution) ACM UbiComp 2012, Pittsburgh · Conference
    In brief

    MIT's Mood Meter counted smiles across campus with cameras, measuring a community's mood in real time.

    PDF
    In the press New ScientistFast Company DesignBostInnoPopular ScienceInnovation News DailyPBS NewsHourMIT NewsMIT NewsNew Scientist
  2. Exploring Temporal Patterns towards Classifying Frustrated and Delighted Smiles E. Hoque, D. J. McDuff, and R. W. Picard IEEE Transactions on Affective Computing 3(3) · Journal · Nominated among the most influential TAC articles, 2015
    In brief

    People often smile when frustrated. By looking at how a smile unfolds over time, AI can tell frustrated from delighted smiles better than people can.

    PDFMIT News
    In the press MIT NewsNPRTimeThe AtlanticWall Street JournalThe TelegraphWiredWiredMIT Technology ReviewMIT Technology ReviewPopular ScienceDaily MailMashableEngadgetBostInnoFast CompanyFast Company
  3. My Automated Conversation Helper (MACH): Helping People Improve Social Skills E. Hoque ACM International Conference on Multimodal Interaction (ICMI 2012), Doctoral Consortium · Workshop, abstract or demo PDF
  4. Mood Meter: Large-Scale and Long-Term Smile Monitoring System J. Hernandez*, E. Hoque*, and R. W. Picard (*equal contribution) ACM SIGGRAPH Emerging Technologies 2012 · Workshop, abstract or demo PDF

2011

  1. Are You Friendly or Just Polite? Analysis of Smiles in Spontaneous Face-to-Face Interactions E. Hoque, L.-P. Morency, and R. W. Picard Affective Computing and Intelligent Interaction (ACII 2011), Memphis · Conference · Oral presentation
    In brief

    Polite and amused smiles look similar but unfold differently. Timing analysis tells them apart.

    PDF
  2. Acted vs. Natural Frustration and Delight: Many People Smile in Natural Frustration E. Hoque and R. W. Picard IEEE Automatic Face and Gesture Recognition (FG 2011), Santa Barbara · Conference · Best Paper Nomination (top 5 of 245)
    In brief

    Many people smile when genuinely frustrated, which acted datasets miss. Best Paper nominee at IEEE FG.

    PDF
  3. Machine Learning for Affective Computing E. Hoque, D. J. McDuff, L.-P. Morency, and R. W. Picard Affective Computing and Intelligent Interaction (ACII 2011), Memphis · Workshop, abstract or demo

2010

  1. I See You: Towards Robust Recognition of Facial Expressions and Speech Prosody in Real Time E. Hoque and R. W. Picard IEEE Computer Vision and Pattern Recognition (CVPR 2010), San Francisco · Workshop, abstract or demo
  2. Design and Evaluation of Interactive, Customizable and Extensible Speech Enabled Games as Speech Therapy for Kids with Autism E. Hoque, M. Goodwin, R. el Kaliouby, and R. W. Picard International Meeting for Autism Research (IMFAR 2010), Philadelphia · Workshop, abstract or demo

2009

  1. When Human Coders (and Machines) Disagree on the Meaning of Facial Affect in Spontaneous Videos E. Hoque, R. el Kaliouby, and R. W. Picard Intelligent Virtual Agents (IVA 2009), Amsterdam · Conference
    In brief

    When human coders and machines disagree about facial expressions in spontaneous video, and what that means for emotion recognition.

    PDF
  2. Exploring Speech Therapy Games with Children on the Autism Spectrum E. Hoque, J. K. Lane, R. el Kaliouby, M. Goodwin, and R. W. Picard INTERSPEECH 2009, Brighton · Conference
    In brief

    Speech-therapy games designed with children on the autism spectrum.

    PDF
  3. An Integrated Approach to Emotional Speech and Gesture Synthesis in Humanoid Robots P. Robbel, E. Hoque, and C. Breazeal Workshop on Affective Interaction in Natural Environments (AFFINE), ICMI 2009 · Workshop, abstract or demo PDF
  4. User-Centered Design of Technology for Just-In-Time, In-Situ Exploration of Facial Affect on the Autism Spectrum M. Eckhardt, M. Madsen, Y. Kashef, A. R. Nasser, E. Hoque, R. el Kaliouby, M. Goodwin, and R. W. Picard International Meeting for Autism Research (IMFAR 2009), Chicago · Workshop, abstract or demo
  5. Lessons from Participatory Design with Adolescents on the Autism Spectrum M. Madsen, R. el Kaliouby, M. Eckhardt, E. Hoque, M. Goodwin, and R. W. Picard ACM CHI 2009, Work-in-Progress, Boston · Workshop, abstract or demo PDF

2008

  1. The Interaction between Information and Intonation Structure: Prosodic Marking of Theme and Rheme M. M. Louwerse, P. Jeuniaux, B. Zhang, J. Wu, and E. Hoque Annual Meeting of the Cognitive Science Society (CogSci 2008), Washington, DC · Conference PDF
  2. Analysis of Speech Properties of Neurotypicals and Individuals Diagnosed with Autism and Down Syndrome E. Hoque ACM ASSETS 2008, Student Research Competition, Halifax · Workshop, abstract or demo · Student Research Competition finalist PDF

2007

  1. What Speech Tells Us about Discourse: The Role of Prosodic and Discourse Features in Dialogue Act Classification E. Hoque, M. S. Sorower, M. Yeasin, and M. M. Louwerse IEEE International Joint Conference on Neural Networks (IJCNN 2007), Orlando · Conference PDF
  2. Multimodal Communication in Face-to-Face Conversations M. M. Louwerse, N. Benesh, E. Hoque, P. Jeuniaux, G. Lewis, J. Wu, and M. Zirnstein Annual Meeting of the Cognitive Science Society (CogSci 2007), Nashville · Conference PDF

2006

  1. Robust Recognition of Emotion from Speech E. Hoque, M. Yeasin, and M. M. Louwerse Intelligent Virtual Agents (IVA 2006), Marina del Rey · Conference · Best Paper Nomination
    In brief

    Recognizing emotion from speech robustly. Best Paper nominee at IVA 2006.

    PDF
  2. Multimodal Communication in Computer-Mediated Map Task Scenarios M. M. Louwerse, P. Jeuniaux, E. Hoque, J. Wu, and G. Lewis Annual Conference of the Cognitive Science Society (CogSci 2006), Vancouver · Conference PDF
  3. 2D Captchas from 3D Models E. Hoque, D. J. Russomanno, and M. Yeasin IEEE SoutheastCon 2006, Memphis · Conference PDF
  4. Robust Recognition of Emotion in e-Learning Environment E. Hoque, M. Yeasin, and M. M. Louwerse 18th Annual Student Research Forum, Memphis · Workshop, abstract or demo · Best Poster Award

2005

  1. Embodied Conversational Agents: Multimodal Communication and Embodiment P. Jeuniaux, M. M. Louwerse, J. Wu, and E. Hoque Garachico Workshop: Symbols, Embodiment and Meaning, Tenerife · Workshop, abstract or demo
  2. Determining the Viability of Electrically Treating 6061 T6511 Aluminum for Use in Manufacturing Processes J. C. Heigel, J. S. Andrawes, J. T. Roth, E. Hoque, and R. M. Ford Transactions of the North American Manufacturing Research Institute of SME (NAMRI/SME) · Journal PDF
  3. Automated Image Analysis of Microstructure Changes in Metal Alloys E. Hoque, R. M. Ford, and J. T. Roth IS&T/SPIE Electronic Imaging: Machine Vision Applications in Industrial Inspection XVIII, San Jose · Conference PDF