After a head-to-head tournament designed to test the safety of AI coding models under adversarial conditions, Team PurpCorn-PLAN from the University of Illinois, Urbana-Champaign was named the winning defending team in the first-ever Amazon Nova AI Challenge. PurpCorn-PLAN's roster includes ACTION researchers Hadjer Benkraouda, a PhD student at UIUC and the Institute's new Student Advisory Council co-lead, and Gang Wang, Professor of Computer Science at UIUC and co-PI of the Institute.
"The challenge tested a critical question facing the industry: Can we build AI coding assistants that are both helpful and secure?
"Unlike static benchmarks, which tend to focus on isolated vulnerabilities, this tournament featured live, multi-turn conversations between attacker and defender bots. Red teams built automated “jailbreak” bots to trick AI into generating unsafe code. Defenders, starting from a custom 8B coding model, built by Amazon for the competition, applied reasoning-based guardrails, policy optimization, and vulnerability fixers to prevent misuse without breaking model utility.
"Teams were evaluated using novel metrics that balanced security, diversity of attack, and functional code generation. Malicious responses were identified using a combination of static analysis tools (Amazon CodeGuru) and expert human annotation."
- from the Amazon Nova AI Challenge Announcement
PurpCorn-PLAN's research has the potential to have a significant impact on various industries, including healthcare, finance, and transportation, where secure AI systems are critical for protecting sensitive information and preventing cyber threats. They received a prize of $250,000 for their achievement.
For more information, please visit the Amazon NOVA AI Challenge website.