Ph.D. Candidate in Computer Science at Northeastern University, advised by Dr. Timothy Bickmore. My dissertation focuses on building: "A Simulation Framework for Assessing Quality of Care & Risk in AI Psychotherapy". My research leverages Artificial Intelligence, particularly in Human-Computer Interaction, Affective Computing, and Natural Language Processing, to create safe and clinical aligned technologies for mental health support.
The team members and I who were working on the PeerSupport project, created our own suicidal ideation test set by self annotating almost 2K pieces of text from Twitter and Reddit (r/SuicideWatch). I performed a grid search on multiple Support Vector Machine hyperparameters. The final accuracy based on the test set came out to be 84%. The most important feature used was TF-IDF (term frequency-inverse document frequency). Each step of the process can be followed through my Google Colab project
Speaker diarization is a key component of systems that support multiparty interactions of co-located users, such as meeting facilitation robots. The goal is to identify who spoke what, often to provide feedback, moderate participation, and personalize responses by the robot. Current systems use a combination of acoustic (e.g. pitch differences) and visual features (e.g. gaze) to perform diarization, but involve the use of additional sensors or require overhead signal processing efforts. Alternatively, automatic speech recognition (ASR) is a necessary step in the diarization pipeline, and utilizing the transcribed text to directly identify speaker labels in the conversation can eliminate such challenges. With that motivation, we leverage large language models (LLMs) to identify speaker labels from transcribed text and observe an exact match of 77% and a word level accuracy of 90%. We discuss our findings and the potential use of LLMs as a diarization tool for future systems.
We introduce a novel application of large language models (LLMs) in developing a virtual counselor capable of conducting motivational interviewing (MI) for alcohol use counseling. Access to effective counseling remains limited, particularly for substance abuse, and virtual agents offer a promising solution by leveraging LLM capabilities to simulate nuanced communication techniques inherent in MI. Our approach combines prompt engineering and integration into a user-friendly virtual platform to facilitate realistic, empathetic interactions. We evaluate the effectiveness of our virtual agent through a series of studies focusing on replicating MI techniques and human counselor dialog. Initial findings suggest that our LLM-powered virtual agent matches human counselors' empathetic and adaptive conversational skills, presenting a significant step forward in virtual health counseling and providing insights into the design and implementation of LLM-based therapeutic interactions.
Games have been successfully used to provide engaging health interventions for adolescents. However, translating health education goals into a playable game has historically taken many person-months of effort, involving game designers, scriptwriters, and artists. This work presents an exploratory study into rapidly developing physician-validated health education games for adolescents using virtual agents and LLMs. We evaluated this approach in an intervention to promote Human Papillomavirus (HPV) vaccination among adolescents, as lack of knowledge and vaccine hesitancy contribute to suboptimal HPV vaccination rates. We conducted a between-subjects randomized study comparing a fantasy narrative game to a non-gamified pedagogical virtual agent, with both interventions conveying the same HPV information. Among our study’s 9-12-year-old adolescent participants, our findings demonstrate large pre-to-post improvements in HPV knowledge for both conditions. The gamified intervention showed higher engagement and entertainment than the pedagogical agent based on participant interviews, demonstrating that gamification enriched the educational experience for adolescents.
We introduce the concept of “empathic grounding” in conversational agents as an extension of Clark’s conceptualization of grounding in conversation in which the grounding criterion includes listener empathy for the speaker’s affective state. Empathic grounding is generally required whenever the speaker’s emotions are foregrounded and can make the grounding process more efficient and reliable by communicating both propositional and affective understanding. Both speaker expressions of affect and listener empathic grounding can be multimodal, including facial expressions and other nonverbal displays. Thus, models of empathic grounding for embodied agents should be multimodal to facilitate natural and efficient communication. We describe a multimodal model that takes as input user speech and facial expression to generate multimodal grounding moves for a listening agent using a large language model. We also describe a testbed to evaluate approaches to empathic grounding, in which a humanoid robot interviews a user about a past episode of pain and then has the user rate their perception of the robot’s empathy. We compare our proposed model to one that only generates non-affective grounding cues in a between-subjects experiment. Findings demonstrate that empathic grounding increases user perceptions of empathy, understanding, emotional intelligence, and trust. Our work highlights the role of emotion awareness and multimodality in generating appropriate grounding moves for conversational agents.
This research explores using Large Language Models (LLMs) to create emotionally realistic simulated patients for counselor empathy training. We introduce SimPatient, a system embodied by the Furhat robot, designed to help novice counselors practice their skills. Two versions were tested: a control with basic dialogue and an intervention with appraisal-based emotion modeling and nonverbal behaviors (NVBs). A study with 6 participants compared perceived realism and impact on counselor empathy. While the emotion model enhanced perceived realism, particularly regarding NVBs, it did not increase empathy and, surprisingly, significantly decreased it in the intervention group. This suggests that adding emotional responses alone may not improve empathy and highlights the complexity of simulating realistic patient interactions, especially with resistant patients. Future research with larger samples and refined implementations is needed.
This research introduces a novel method for training specialized dialogue agents. Leveraging separate Large Language Models (LLMs) for user simulation and reward modeling, reinforcement learning from AI feedback (RLAIF) provides automated, scalable feedback, eliminating the reliance on scarce and costly human input. This, combined with parameter-efficient fine-tuning (PEFT) via LoRA, enables the efficient training of highly effective, adaptable, and robust dialogue agents for diverse applications where collecting human feedback is impractical.
September 2022 - Present
January 2019 - May 2020
May 2017 - December 2018
May 2017 - August 2017
May 2017 - August 2017
