# Validation That Inspects Steps Cannot See Composition ## Claim A safeguard that evaluates each request in isolation is blind to attacks assembled from individually legitimate requests, so filtering has to operate on the sequence rather than the item. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Holds wherever an attack can be decomposed into steps each of which has legitimate uses on its own — deanonymization via summarize-search-rank, API request chains, batch request routing. - Breaks where any single step is itself uniquely diagnostic of malicious intent, regardless of context. - Breaks where the composing party is identifiable and trackable across steps, so the sequence can be reconstructed after the fact even if it wasn't blocked in the moment. ## Evidence A research paper on LLM-enabled deanonymization argues provider-side guardrails are unlikely to stop the attack, because its four-stage pipeline (Extract, Search, Reason, Calibrate) decomposes into individually benign-looking summarization, search, and ranking requests. No single request in the chain resembles an attack; the attack exists only in how the steps compose. > [!quote] > we are pessimistic that this is possible as our deanonymization framework splits an attack into a combination of seemingly benign summarization, search and ranking tasks. ## Links - [[Commons & Community Governance MOC]] — why per-request safeguards miss attacks assembled from legitimate parts - [[3. Resources/Literature Notes/Large-scale Online Deanonymization with LLMs]] — source paper - [[3. Resources/MOCs/AI Agent Safety MOC]] — because applies-in; a general principle for designing AI guardrails and content-safety systems - [[3. Resources/MOCs/WordPress Hosting Security Stack MOC]] — because extends; the same structural flaw (step-level validation missing composed attacks) recurs in the WP2Shell route/validation desynchronization pattern already in the vault ## Tests / falsifiers - Submit both a decomposed and a single-shot version of the same deanonymization workflow to a major provider and measure detection/refusal rates — if the decomposed version is caught at the same rate as the single-shot version, the composition-blindness claim doesn't hold for that provider. - Find a guardrail system that successfully flags composed attacks by tracking request sequences rather than individual requests — that would be the working counterexample/mitigation, and worth documenting as the fix. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Literature Notes/Large-scale Online Deanonymization with LLMs]] — arXiv preprint, ETH Zurich / MATS / Anthropic