Snyk Research Shows AI Agents Can Fix Vulnerable Dependencies With High Accuracy
New benchmark tests prove AI agents can upgrade vulnerable packages without breaking builds, but they require security data to succeed.
Key points
- Snyk’s VulnBump benchmark tests AI agents on 307 real-world dependency upgrades across six package managers.
- Claude Sonnet 5.5 powered agents passed 90.2% of tasks when given Snyk scan data, compared to 59.6% without it.
- Keeping builds green was easy for agents; accurately removing vulnerabilities proved to be the primary challenge.
Snyk published new research demonstrating that AI agents can successfully fix vulnerable software dependencies without breaking application builds, provided they receive accurate security data. The study, titled VulnBump, evaluated how well automated tools handle the complex task of upgrading packages to patched versions while maintaining code stability.
The benchmark tested three approaches using the Claude Sonnet 5.5 model on 307 real upgrade tasks. Snyk’s purpose-built Evo Remediation Agent achieved a 90.2% pass rate. Anthropic’s Claude Code reached 77.5% when provided with Snyk’s recommended upgrades and scan results, but dropped to 59.6% when operating without that security intelligence. A non-AI baseline that simply applied recommended versions passed 46.9% of tasks.
Context and Background
Snyk Open Source recorded 459 million vulnerability findings across approximately 300,000 organizations over six months in 2026. Although 94% of these vulnerabilities had known fixed versions, only about 10% were resolved. The difficulty lies in upgrading safely, as many packages are transitively imported and major version bumps often introduce breaking changes. Even minor releases sometimes contain incompatibilities.
This research follows Snyk’s June VulnBench study, which found that combining Claude and Snyk Code improved security reviews. VulnBump extends this finding to remediation, showing that adding Snyk’s scan data increased Claude Code’s pass rate by 17.9 percentage points. This points in the same direction as previous findings regarding the combination of AI models and security tools.
Who Is Affected
The study covers Maven, Gradle, npm, Yarn, pip, and Poetry, which handle about 92% of production findings. Researchers selected 43 repositories from 767 candidates to match production data on ecosystem, severity, and dependency type. Each task required the agent to install dependencies, build the project, pass tests, and eliminate vulnerabilities.
Agents struggled to verify fixes. In 70 of 124 failures, unaided Claude Code claimed vulnerabilities were fixed, but rescans proved they remained. All three agents kept builds green 93% of the time, indicating that maintaining code functionality is easier than ensuring security patches are correctly applied and verified.
What should you do?
Organizations using AI for dependency management should verify that their tools integrate with accurate vulnerability databases. Relying on models without security context leads to false confidence in remediation efforts. Teams should treat AI-generated upgrades as drafts requiring human or automated verification through scanning tools to confirm vulnerabilities are actually removed.
What to do and how to stay safe: Snyk
- Verify that AI remediation tools integrate with real-time vulnerability data rather than relying solely on model training.
- Implement automated rescans after any AI-driven dependency upgrade to confirm vulnerabilities are actually removed.
- Monitor build logs and test results to ensure that security updates do not introduce breaking changes or instability.
- Review transitive dependencies manually, as AI agents may miss indirect vulnerabilities that persist after direct package updates.
Step-by-step guide: Risk-Based Vulnerability Management: Prioritize Fixes by Actual Danger
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Can AI agents fix vulnerabilities without breaking builds?
Yes, but accuracy depends heavily on whether the agent receives external security data; pass rates range from 59.6% to 90.2%.
Why do so many vulnerabilities remain unfixed despite available patches?
Upgrading safely is difficult because many packages are transitive and version bumps often introduce breaking changes that break builds.
How was the VulnBump benchmark conducted?
Researchers tested agents on 307 tasks from 43 real repositories across six package managers, verifying fixes via installation, builds, tests, and scans.
