It’s an axiom that today’s cybersecurity teams know all too well: People are more likely to trust communications that appear to come from people they already know. Yet more and more of those communications now pass through artificial intelligence (AI). Emails, documents, presentations, and messages are drafted and edited using AI. In today’s online ecosystem of ever-improving large language models and a growing amount of personal data, people are increasingly interacting with one another through AI agents that can act on someone’s behalf.
But what if an AI agent could stand in for a specific individual? Early research suggests that AI agents modeled on a real person can create a stronger sense of trust and presence than generic AI assistants. This points to an opportunity for organizations to improve workflows and efficiency by building AI that makes decisions and communicates just like a real coworker would.
“We’re essentially talking about creating AI agents that represent a specific person, so it definitely raises serious questions about privacy, consent, and misuse,” said Diego Gómez-Zará, assistant professor of computer science and engineering at the University of Notre Dame. “These agents are already being developed, but we know very little about how people relate to them. We need to understand how these technologies can be built safely, securely, and in a way that people will accept.”
These are the questions that Gómez-Zará will be pursuing for the next three years thanks to a new $450,000 Air Force Office of Scientific Research Young Investigator Program award. The program is intended to enhance the early-career development of outstanding young researchers while pursuing Air Force mission and related challenges in science and engineering.
The Air Force is interested in these types of AI proxies to help preserve an Airman’s knowledge and decision-making abilities during transitions and personnel rotations, especially in leadership positions. There is also potential to help move projects and missions forward when lines of communication are sporadic or cut off.
With the new support, Gómez-Zará will take a multipronged approach to answering a multitude of questions related to this intriguing possibility. For example: What makes people trust a personal AI agent enough to delegate decisions to it? Will colleagues trust an agent acting on someone’s behalf, or prefer the real person? How does AI-mediated interaction affect trust among the other people involved?
“At the heart of these questions is a concept that I have been developing called ‘proxy trust,’” said Gómez-Zará. “That is the degree to which the trust we place in a colleague carries over to the AI agent that represents them. Understanding when that trust is warranted and building agents that protect people’s privacy is the primary goal of this work.”
Over the coming years, Gómez-Zará and his research team will run multiple experiments to answer these questions. Participants will interact with AI agents that resemble real people to different degrees to help find a balance between imitation and automation. Longitudinal studies over several weeks will track how working relationships develop between these AI proxies and human colleagues. Paired studies between people who know one another will further help facilitate the research.
On the technical side of the equation, researchers will also explore recommendations for data handling, model safety, and large-scale deployment guardrails built into the code. At the conclusion of the project, Gómez-Zará hopes to provide frameworks and design recommendations to help organizations build AI agents people can trust, with data privacy and safety built in from the start.
“As companies and organizations deploy more AI agents, they are inevitably going to become a layer within some of our social and professional interactions,” said Gómez-Zará. “I want to help us understand the consequences of these use cases to make sure they’re helping people instead of hurting them.”