Enabling and Evaluating Human-Agent Collaboration
Diyi Yang
Computer Science
Stanford University
Feb 10 @ 12:00 pm PT
Zoom link to be made available soon.
Abstract:
Recent advances in large language models (LLMs) have revolutionized human-AI interaction, but their success depends on addressing key challenges like privacy and effective collaboration. In this talk, we first explore PrivacyLens, a general framework to evaluate privacy leakage in LLM agents’ actions, by extending privacy-sensitive seeds into agent trajectories. By evaluating state-of-the-art models, PrivacyLens reveals contextual and long-tail privacy vulnerabilities, even under privacy-enhancing instructions. We then introduce Co-Gym, a novel framework for studying and enhancing human-agent collaboration across various tasks. Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance. Via PrivacyLens and Co-Gym, this talk highlights how to develop AI systems that are trustworthy and capable of fostering meaningful collaboration with human users.
Biography:
Diyi Yang is an assistant professor in the Computer Science Department at Stanford University. Her research focuses on human-centered natural language processing and computational social science. She is a recipient of Microsoft Research Faculty Fellowship (2021), NSF CAREER Award (2022), an ONR Young Investigator Award (2023), and a Sloan Research Fellowship (2024). Her work has received multiple paper awards or nominations at top NLP and HCI conferences.