Trust plays a fundamental role in shaping the willingness of users to engage and collaborate with AI systems. Yet, measuring user trust remains challenging due to its complex and dynamic nature. While traditional survey methods provide trust levels for long conversations, they fail to capture its dynamic evolution during ongoing interactions. Here, we present VizTrust, which addresses this challenge by introducing a real-time visual analytics tool that leverages a multi-agent collaboration system to capture and analyze user trust dynamics in human-agent communication. Built on established human-computer trust scales—competence, integrity, benevolence, and predictability—, VizTrust enables stakeholders to observe trust formation as it happens, identify patterns in trust development, and pinpoint specific interaction elements that influence trust. Our tool offers actionable insights into human-agent trust formation and evolution in real time through a dashboard, supporting the design of adaptive conversational agents that respond effectively to user trust signals.
A front-end chatbot of VizTrust (set up with Llama-3.1-8b model) interacts with human users in natural languages on topics related to conversational agent design. As receiving each user’s utterance, the front-end chatbot passes the current conversation transcript to a multi-agent system that evaluates trust and Python programs for user behavior analysis using natural language processing and machine learning techniques.
In the visualization dashboard interface, the main view presents four time series visualizations of all conversation turns, including user trust dynamics, user engagement, user emotional tones, and user theory of politeness. Each visualization plot has interactive features that allow design stakeholders to gain more detailed information on trend-changing points. The design stakeholders can select any conversation turn on the plot of user trust dynamics to read supporting evidence on the trust rating scores in detail.
We thank the reviewers for their constructive feedback on this research.
@inproceedings{wang2025viztrust,
title={VizTrust: A Visual Analytics Tool for Capturing User Trust Dynamics in Human-AI Communication.},
author={Wang, Xin and Tulk Jesso, Stephanie and Kojaku, Sadamori and Neyens David M. and Kim, Min-Sun},
booktitle={Extended Abstracts of the CHI Conference on Human Factors in Computing Systems},
pages={1--10},
year={2025}
}If you have any questions, please feel free to contact Vision(Xin) Wang