Beyond the Lab: Securing Learning Systems in the Wild
Yigitcan Kaya
SecLab
UC Santa Barbara
5 February 2026 @ 2:00 pm PST
Zoom: https://ucsb.zoom.us/j/87605586897?pwd=OSzSOE8UDAZgurC3acnzFdZlsWKpln.1
Abstract:
AI systems have rapidly become part of critical infrastructure, powering malware detection, vulnerability analysis, and customer-facing services. While research benchmarks show impressive progress, this progress has not consistently translated into reliability in real deployments, where systems face adversaries, distribution shift, and operational constraints. My research sits at the intersection of AI and systems security and is driven by a simple question: why do models that look strong in the lab fail in the wild? Using large-scale empirical studies of deployed systems, I uncover previously unknown failure modes and violated defensive assumptions, and design practical, deployment-grounded defenses.
This talk presents two recent case studies from different domains that expose a shared root cause of insecurity: defenses that appear sound in isolation collapse when their underlying assumptions are violated in practice. First, I examine the fast-growing ecosystem of web-based AI chatbots built on third-party plugins. I show that widespread implementation flaws invalidate model-level prompt-injection defenses, enabling attackers to inject privileged instructions and achieve orders-of-magnitude higher success than anticipated in standard evaluations. Second, I study security classifiers such as malware detectors, which are conventionally evaluated under the assumption that every input has a single, correct label. In reality, labels are not intrinsic properties of inputs, but encode organizational policy and risk tolerance. I formalize this mismatch as policy discrepancy, show that it is a major and overlooked source of deployment failure, and introduce a framework for adapting models to their operational context using post-deployment feedback. I conclude by outlining future directions, including leveraging AI vulnerabilities as proactive defenses against offensive AI, and building world-model–based security reasoning to enable more reliable AI in deployment.
Biography:
Yigitcan Kaya is a postdoctoral fellow at UC Santa Barbara, where he works with Giovanni Vigna and Chris Kruegel. He received his Ph.D. from the University of Maryland, College Park. His research lies at the intersection of trustworthy AI and systems security, with a focus on understanding how AI systems behave under the real-world conditions of security-critical applications. His work has appeared at flagship venues in machine learning (ICLR, ICML) and security (IEEE S&P, USENIX Security), and has been covered by outlets including MIT Technology Review and VentureBeat. His research is supported by a U.S. Intelligence Community Postdoctoral Fellowship and an Amazon Research Award.