How Older Adults Evaluate AI Chatbots
Generative AI / Trust / Accessibility
I led an interview study to understand how older adults use AI chatbots, what builds or reduces trust, and what product teams can do to support them.
Project Objective
Generative AI systems are moving into everyday life, but older adults are often treated as secondary users or framed through deficit-based assumptions. For teams, this creates a risk: AI systems may be technically powerful yet fail on onboarding, credibility, privacy, and accessibility for a major user group.
The core question was practical: after a first hands-on experience with an LLM chatbot, what makes older adults see it as useful, trustworthy, risky, or worth adopting?
Project Approach
Across the project, we collected more than 18 hours of audio. The analysis moved from 38 initial codes to a refined 27-code codebook, then into four higher-level themes that translate directly into design strategy.
Timeline
* Human-subjects IRB approval extended the calendar timeline; once approval was in place, interviews and analysis moved quickly.
Tools
Interview workflow
Used Otter.ai for transcription support and MAXQDA for coding, codebook refinement, and thematic analysis.
Guided exploration
Introduced participants to ChatGPT 3.5 with neutral prompts before open-ended exploration.
Analysis structure
Moved from 38 initial codes to a refined 27-code codebook before synthesizing themes.
Session capture
Collected more than 18 hours of interview audio and paired observations with participant explanations.
Method Alternatives and Choice
Alternative: Survey
A survey could have measured attitudes at scale, but it would have missed how participants formed opinions while actually trying the chatbot, especially when confusion, delight, privacy concern, or credibility checking happened in the moment.
Alternative: Task-based usability test
A narrow usability test could have identified interface breakdowns, but the main question was broader: how older adults make sense of an unfamiliar AI system, what they trust it with, and where they see value in daily life.
Alternative: Longitudinal diary approach
A diary approach could show adoption over time, but it would require participants to already know enough to use the tool independently. That would bias the project away from first impressions and early onboarding barriers.
Chosen: Contextual interviews
Contextual interviews let me observe first-use behavior, capture self-directed prompts, and immediately probe the reasoning behind trust, usefulness, privacy boundaries, health-information expectations, and social-companion reactions.
Representative Data
"This is personal. It directly answered my question just as I was talking to you."
Walter, on chatbot responses compared with search
"I would not be using [ChatGPT] for a doctor's advice."
Beatrice, on health advice needing verification
"I have no clue who owns this software, who controls it."
Richard, on privacy and data handling uncertainty
"Voice could be good, maybe because I was having difficulty with my manual typing."
Richard, on accessible interaction
Key Insights
Participants saw the chatbot as an expert, a neutral diplomat, or a potential companion.
These mental models shaped whether they expected the system to provide facts, balanced perspectives, emotional support, or expert advice.
Fast, personal answers were compelling, but source credibility was a deciding factor.
Participants appreciated responses that felt tailored, but wanted recognizable sources, especially for unfamiliar or higher-stakes topics.
Privacy uncertainty restricted what people were willing to share.
When participants did not know how prompts were stored or used, they avoided personal details that would have made the chatbot more helpful.
Voice and multimodal responses matter, but the format should match the task.
Voice could make interaction easier, while visual aids, videos, or printable summaries may be better for instructions that users need to revisit.
What This Means for Teams
Design transparent trust cues
Show where information comes from, explain confidence and limitations, and distinguish general information from advice that requires expert verification.
Make onboarding visible and low-friction
Provide prompt examples, clearer send affordances, and first-run guidance designed for users who may not already know AI interaction patterns.
Offer privacy controls in plain language
Explain what is stored, who can access it, whether data trains models, and how users can keep conversations private.
Use task-aware multimodality
Pair speech with text, summaries, videos, or step-by-step artifacts depending on whether the user is exploring, deciding, remembering, or acting.
Outcomes
Publication: This work was published in International Journal of Human-Computer Interaction in 2025. View paper.