Intruder AI Pentester Finds Auth Bypass Missed by Humans in Five Years

Intruder reported that its artificial intelligence penetration testing agents identified a complex authentication bypass vulnerability. This flaw had remained undetected in the target codebase for five years despite multiple human-led penetration tests. The discovery highlights a specific strength of automated AI agents in uncovering subtle logic errors that manual testers often overlook during standard assessment cycles.

According to BleepingComputer, Intruder engineers found that rigid adherence to traditional testing methodologies often hinders AI performance. When agents were forced to follow strict structures looking for specific vulnerability types, they missed issues outside their narrow remit. The company observed that these constrained agents would miscategorize findings or ignore entirely new vulnerabilities, resulting in confusing and incomplete security reports for clients.

Methodology and AI Limitations

Intruder determined that a complete lack of structure also produced poor results. Unrestricted AI agents struggled to manage large applications effectively. While free agents could discover interesting anomalies, they frequently missed significant numbers of other vulnerabilities within the targets. The inconsistency in coverage made unguided testing unreliable for comprehensive security assessments.

The company settled on a balanced approach that gives agents freedom with broadly defined roles. This method allows the AI to approach and manage large applications effectively. Simultaneously, Intruder implemented stricter guardrails to prevent the agents from causing damage during the testing process. This hybrid model aims to maximize discovery while maintaining operational safety.

Human vs. Automated Testing

Traditional human pentesting relies on methodology documents that provide assurance of comprehensive testing. Clients often expect a checklist approach where every specific item, such as unauthenticated SQL injection, is explicitly tested. This creates a perception that manual testing is a perfectly repeatable product where any tester can follow the same steps to find everything important.

Intruder notes that this perception is misleading because penetration testing is a skilled trade. The most impactful findings in human tests often come from experienced testers noticing weird behaviors and following up on them intuitively. However, even skilled humans can miss persistent flaws, as evidenced by the five-year-old authentication bypass that the AI agents successfully identified.

What should you do?

Organizations using penetration testing services should evaluate how their providers integrate automated tools. Understanding the balance between AI discovery and human review is essential for accurate risk assessment. Security teams should not assume that strict methodology checklists guarantee the discovery of all logical vulnerabilities.

What to do and how to stay safe: Intruder

• Review your current penetration testing methodology to ensure it includes automated logic testing capabilities.
• Compare findings from human-led tests with automated scans to identify potential blind spots.
• Verify that AI testing tools have proper guardrails to prevent damage to production environments.
• Request detailed reports that explain how vulnerabilities were discovered, not just what was found.

Step-by-step guide: Risk-Based Vulnerability Management: Prioritize Fixes by Actual Danger

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Can AI pentesting replace human pentesters entirely?

No, Intruder notes that AI has limitations in consistency and risk assessment, and human testers provide valuable context and experience.

Did the AI agent find more vulnerabilities than the humans?

The source material only confirms that the AI found one specific authentication bypass that humans missed over five years.

How does Intruder’s AI pentester operate?

It uses broadly defined roles with freedom to explore, combined with stricter guardrails to prevent damage, avoiding strict vulnerability-type checklists.