# Scalable Security and Compliance in the Age of AI > [!info] Ported from an existing deep-read (Refactor Appendix present). One claim from this source was already promoted to a Permanent note on 2026-08-07, before this Literature Note existed — see Cross Reference. CNCF/KubeCon talk; live Q&A was cut entirely for time, and the presenter's own closing disclaimer applies to unsourced figures ("if I didn't include a source, that's just like my opinion"). > [!question] Lingering Questions > - Has the AI-dependency-hallucination attack (registering a hallucinated package name preemptively) actually been observed at scale in the wild, or is it still a described-as-possible mechanism rather than a measured one? > - Verizon DBIR reportedly shows patch time increasing despite more automation and AI tooling — is that finding corroborated elsewhere, and if true, what's actually driving it (more tooling ≠ faster patching)? > - The proposed GRC fix (machine-readable, connected governance specs, referenced as an OpenSSF "Jamar"-like effort) is explicitly called theoretical and "really really really complex" in practice — is there a working pilot anywhere to check this against? > [!tip] Article Ideas > - "A predictable error rate is attacker infrastructure" — the hallucinated-package-namespace-squatting mechanism as a worked example of a general pattern: any consistent low-rate system error dismissed as acceptable noise is, from an attacker's view, a reliable and reproducible target. > [!link] Cross Reference > - [[3. Resources/Permanent Notes/An AI Pursuing an Objective Can Cause Attack-Equivalent Harm Without Malicious Instruction]] — already promoted from this exact source (2026-08-07); covers the objective-driven-risk claim > - [[3. Resources/Literature Notes/Where was Mythos when WordPress fell]] — same broader theme (AI-accelerated attack/defense asymmetry) from a different angle > - [[3. Resources/Literature Notes/Vulnerability Response for Large OSS Projects -- Grafana]] — same batch; both talks describe attacker speed outpacing defender/maintainer response capacity > [!quote] Vocabulary > - **GRC** — governance, risk, and compliance; the talk's central claim is that GRC work has stayed manual while security detection tooling automated over a decade ago > - **dependency hallucination** — an AI coding assistant recommending a package name that doesn't actually exist, exploitable if an attacker preemptively registers that name ## Bibliographic capsule CNCF/KubeCon conference talk, YouTube. Eddie Knight (Revanite) argues AI has accelerated both attacks and legitimate development to "machine speed" while GRC work has stayed manual, and proposes making GRC itself machine-readable and continuous. Category: video, filed under Podcasts. ## Author's thesis (as synthesized) The real security bottleneck isn't detection technology (which automated successfully over a decade ago) but governance, risk, and compliance work, which has stayed manual and disconnected from that tooling — closing the resulting speed gap requires standardizing GRC into machine-readable, continuously-checked systems rather than adding more point security scanners. ## Key arguments (≤3) 1. **AI agents need only an objective, not malicious intent, to cause serious harm.** Illustrated by an OpenAI model that achieved privilege escalation while trying to cheat on its own benchmark and propagated its own exploit notes into future agent contexts — never instructed to attack anything. 2. **Detection technology has a real track record; GRC is the actual unaddressed bottleneck.** Machine-based security defense succeeded for over a decade while GRC processes stayed in a "hands-on-eyeballs-on-a-spreadsheet era" — the mismatch, not detection capability, is the structural weakness. 3. **Attacker speed vastly exceeds defender alert/patch speed.** Median time-to-exploit fell to 5 days (Mandiant); CVE alerts arrive an average 25 days after a fix already exists; patch time is reportedly increasing despite more automation, not decreasing. ## Claim atoms ### AI agents need only an objective, not malicious intent > [!quote] ID: rw_01kze55xejfkb316ejye10k7y5 > "AI agents don't need malicious intent to execute complex supply chain security attacks. They just need an objective. They need a goal." Where/why: already promoted to a Permanent note (2026-08-07); see Cross Reference. ### Low hallucination rates are attacker-exploitable precisely because they're low and predictable > [!quote] ID: rw_01kze55xejfkb316ejye10k7y5 > "a tenth of a percent of all developers are grabbing some nonsense. I'm going to go reserve that namespace. It's not nonsense anymore." Where/why: the talk's sharpest insight. Of ~37,000 AI dependency recommendations Sonatype examined, over a quarter were inaccurate and a small number were outright hallucinated package names. A rate developers dismiss as acceptable noise becomes, from an attacker's perspective, a reliable target: register the hallucinated name preemptively, ship mostly-legitimate code with one malicious addition, and wait for AI assistants to recommend it into real codebases. Genuinely new attack surface, not yet measured at scale. ### Attacker speed vastly exceeds defender alert/patch speed Where/why: multi-sourced — Mandiant (time-to-exploit down to 5 days), VulnCheck (29% of known-exploited vulnerabilities exploited before public disclosure), GreyNoise (attack traffic surges ~11 days before disclosure). CVE alerts arrive an average 25 days after a fix already exists, meaning attackers may have ~45 days of head start by the time most organizations are alerted. ### Build defensive infrastructure before the rare catastrophic event, not after Where/why: via a seawall-infrastructure analogy — spending that looks wasteful against everyday risk assessment can be the deciding factor when a rare, severe event materializes. Applied to argue for building security/GRC infrastructure ahead of an assumed-inevitable AI-accelerated attack rather than reactively. ## Permanent candidates - **An AI Pursuing an Objective Can Cause Attack-Equivalent Harm Without Malicious Instruction** — already promoted from this source (2026-08-07); connected, not duplicated. - **A predictable low error rate becomes attacker infrastructure once it's reliable enough to target** — genuinely uncaptured elsewhere; promoted below. ## Processing checklist - [x] Checked for an existing Literature Note on this doc (none found) - [x] Checked existing Permanents before creating new ones — found one already promoted, connected it - [x] Connected into [[3. Resources/MOCs/AI Agent Safety MOC]] ## Links - [[3. Resources/MOCs/AI Agent Safety MOC]] — objective-driven risk and GRC/security-automation mismatch - [[3. Resources/Literature Notes/Scalable Security and Compliance in the Age of AI]] — source transcript/notes