53.9% of AI-Generated Security Patches Fail or Break Something New
AI now finds zero-days faster than defenders patch them. 1Password found 53.9% of 6,080 AI-generated patches failed to fix the flaw or introduced new ones. The remediation gap is the risk.
By Grace Kim
3 min read

What's News
- 53.9% of 6,080 AI-generated patches failed to fix the flaw or introduced a new one, per 1Password research.
- Zhipu AI's open-weight GLM-5.3 built working exploits in 50 of 410 ExploitBench attempts; Claude Mythos Preview hit 56, per Anthropic.
- PHP2Shell flaws CVE-2026-109618 and CVE-2026-109619 were both rated CVSS 9.2; AI helped one researcher find them in about a week.
- A supply chain attack hit LiteLLM 1.107.0 with an infostealer and RAT in the published package while the repo stayed clean.
- Attack attempts were observed before PHP patches became available.
AI can find zero-day vulnerabilities faster than defenders can patch them — and more than half the fixes machines write don't work. That's the uncomfortable math now facing security teams.
Research from 1Password examined 6,080 AI-generated patches and found that 53.9% failed to resolve the original vulnerability, introduced a new security flaw, or both. The results reflect specific testing conditions, but they point to a widening gap: the industry is accelerating vulnerability discovery while the work of turning discoveries into protection stays slow.
How fast can frontier models find exploits?
Faster than almost anyone expected. According to research from Anthropic, Zhipu AI's open-weight GLM-5.3 developed working end-to-end exploits in 50 out of 410 attempts on the ExploitBench benchmark. Anthropic's Claude Mythos Preview managed 56 — an extremely small gap between a restricted frontier model and one whose weights anyone can download.
Capabilities that once required highly specialized security researchers are becoming widely accessible. The software underneath hasn't gotten less vulnerable.
What did the PHP2Shell case actually expose?
The PHP2Shell vulnerabilities, researched by Wiz, Calif and OpenAI's Codex Security team, showed frontier AI helping a single researcher uncover critical flaws in the PHP engine in roughly a week. The two vulnerabilities — CVE-2026-109618 and CVE-2026-109619, both rated CVSS 9.2 — were memory-safety flaws in PHP itself, a runtime underpinning a substantial share of the web, including WordPress, Drupal and Joomla. Researchers demonstrated remote code execution and memory disclosure against widely used applications; even securely coded apps were exposed because the weakness sat in the runtime.
The timeline is the revealing part. After an early public warning, PHP's security team accelerated its release schedule — but by the time patches shipped, early attack attempts were already being observed.
The same day, a separate supply chain attack compromised LiteLLM version 1.107.0, a popular open-source gateway for large language models. Attackers embedded an infostealer and a remote access trojan in the published Python package while the source repository stayed clean. A team reviewing source code alone would have missed the payload entirely.
Why does the old playbook break down?
Traditional vulnerability management runs on a familiar loop: discover, report, assign a CVE, patch, scan, prioritize, deploy. The whole security ecosystem was built around it — and it depends on knowing the vulnerability exists in the first place.
Scanners flag known vulnerabilities. They say nothing about flaws without a CVE, an advisory or a patch. CISA's Known Exploited Vulnerabilities Catalog helps prioritize known threats but can't surface undisclosed ones. PHP2Shell proved serious flaws can live in foundational software for years; when frontier AI can uncover them in days, relying exclusively on published advisories means defenders always react to someone else's timeline.
What has to change?
The answer isn't telling overloaded teams to process more alerts. Four shifts matter:
- Treat the CVE database as partial, not complete. Combine scanning with proactive research, code analysis, threat intelligence and a clear inventory of software components.
- More findings don't equal better security. Validate whether issues are reproducible, which versions are affected, whether there's a realistic attack path and what the business impact is — not severity scores alone.
- Make fixes deployable, not just correct. Enterprises run legacy systems and locked dependency versions. NIST's enterprise patching guidance — identify, prioritize, install, verify — needs to move fast enough to keep pace with AI-driven discovery.
- Measure time-to-protection. Track the full lifecycle: validation, patch development, compatibility testing, deployment. At Echo, fixed PHP packages shipped right after upstream patches landed, with updated images for supported PHP versions rebuilt the same afternoon.
Finding a bug was never the hard part's end — proving it's real, patching it correctly and deploying it across hundreds of environments is. As the window between discovery and exploitation shrinks, the next era of vulnerability management belongs to whoever can convert discoveries into reliable, deployable fixes before attackers act.
Source: unite.ai
More from Grace Kim
Show full bio
Market editor covering people, launches and day-to-day practice in entrepreneurship, startups and venture funding at Business Bearings.
2,419 articles