The Prompt as a Human-AI Security Boundary: Predicting and Reducing Vulnerabilities Before AI-Assisted Code Generation
Chaoran Chen ⋅ Ningzhi Tang ⋅ Alfonso A Moreno ⋅ Yanfang (Fanny) Ye ⋅ Yaxing Yao ⋅ Toby J Li
Abstract
Can vulnerable AI-generated code be anticipated before any code is generated? We study this question in real-world AI coding workflows, where developers often specify functional goals while leaving security requirements implicit. We frame this as a \emph{human-AI specification problem}: security is not absent because developers do not care, but because security constraints are often unstated and silently delegated to the model. Analyzing $43,195$ code-generating turns across $12,746$ conversations from Cursor, Copilot, Claude Code, Codex, and Gemini CLI, we find that prompt-level factors predict vulnerable outputs before generation with AUC=$0.79$. Controlled experiments further show that prompt framing affects security outcomes: underspecification significantly increases vulnerability density, security omission shows a consistent directional effect, and urgency leads models to generate shorter outputs that omit security-relevant controls. We introduce SecNudge, a proof-of-concept prompt-time scaffold that selectively surfaces missing security constraints when functional intent and security relevance diverge, reducing vulnerability density by $55\%$ ($p<0.001$). These results shift the focus from expecting developers to write security-perfect prompts toward designing coding tools that scaffold security at the moment missing constraints become consequential.
Chat is not available.
Successful Page Load