JFrog Research Highlights Governance Gaps in AI-Driven Software Development Life Cycles
Autonomous AI agents create ephemeral code changes that evade traditional audit trails, creating new security risks for software teams.
Key points
- AI agents act as autonomous team members but lack persistent accountability due to ephemeral reasoning.
- Traditional git logs fail to capture agent tool calls, model versions, or decision logic.
- New regulations like the EU AI Act mandate logging for high-risk AI systems.
JFrog published research on governing the agentic software development life cycle, stating that current standards cannot keep pace with AI agents that create pull requests and execute infrastructure changes. These autonomous tools operate across the entire SDLC but vanish after sessions end, leaving no trace of their reasoning or decisions for auditors to review.
The company noted that git logs were designed for human commits and show only final code states. They do not record which AI model generated the code, which tools were invoked, or whether any human was accountable for the outcome. This gap breaks the core assumption of existing governance frameworks that a human engineer made every decision.
Challenges in Traditional Governance
Traditional SDLC trust relies on human peer reviews and traceable intent linked to identities and tickets. JFrog’s analysis indicates that volume overload leads to rubber-stamping, where humans approve thousands of lines of agentic code per hour due to fatigue. Agents also introduce non-deterministic decisions and hallucinations that can embed latent vulnerabilities passing functional tests.
Agents are ephemeral, meaning their reasoning and prompt iterations disappear once the session ends. Git commit logs capture only the final state, omitting details about Model Context Protocol servers queried or specific tool calls made. This lack of provenance makes it impossible to hold the process accountable or trace security-critical actions back to a source.
Regulatory and Supply Chain Impacts
The agentic SDLC introduces new supply chain attack surfaces, including prompt injections, malicious MCP tools, and jailbreaks. JFrog stated that these new actors—agents, harnesses, and LLMs—create vectors that traditional security controls were not built to address. Compliance frameworks like SOC 2 and FedRAMP were formalized before this era and focus primarily on human oversight.
Regulations are adapting to these risks. The EU AI Act requires high-risk AI systems to allow automatic event recording and effective human oversight during use. Article 12 mandates log retention over the system’s lifetime, while Article 14 ensures natural persons can oversee operations. OWASP’s AI Security Verification Standard also requires logging AI interactions with session context and security-critical proactive actions.
What should you do?
Teams should attach provenance data directly to SDLC entities like commits or artifacts to enable automatic release gating. JFrog suggested this practice aligns with the security-by-design expectations of the EU Cyber Resilience Act and ISO/IEC 42001 governance approaches. This attachment supports post-release troubleshooting and downstream safety checks by preserving context that would otherwise be lost.
Background: Prompt injection attacks
Look for sudden shifts in tone, refusal to follow system constraints, or output that repeats hidden instructions. These signs indicate the model is processing injected commands rather than your intended query.
Read the full guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
What to do and how to stay safe: JFrog
- Review current SDLC audit trails to identify gaps where AI agent actions are not logged.
- Implement logging mechanisms that capture session context and AI-specific telemetry for all automated code changes.
- Verify that peer review processes include checks for agent-generated code rather than relying on automatic approvals.
- Monitor for compliance with emerging standards like the EU AI Act and OWASP AISVS regarding high-risk AI system logging.
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Why do traditional git logs fail to track AI agent activities?
Git logs were designed for human commits and only show the final code state, missing the agent’s reasoning, tool calls, and model versions.
What new attack vectors do AI agents introduce in the SDLC?
Agents introduce risks such as prompt injections, malicious MCP tools, misaligned models, and jailbreaks that bypass traditional human-centric controls.
How does the EU AI Act address AI system governance?
The act requires high-risk AI systems to allow automatic event recording and ensure they can be effectively overseen by natural persons during use.
