Reducing Cost and Improving Performance With Claude Platform ![rw-book-cover](https://pbs.twimg.com/profile_images/2044472418815893504/xf14RxM8.png) 🔗 **Source:** [Reducing Cost and Improving Performance With Claude Platform](https://twitter.com/ClaudeDevs/status/2097369738968195513/?rw_tt_thread=True) > [!note]+ Document Notes > tags: [prompt-caching, llm-cost-optimization, prompt-engineering, ai-agents, developer-tools] > author: Lance Martin, Brad Abrams, Isabella He, Ben Lehrburger > source: Anthropic / Claude Blog (posted via @ClaudeDevs on X) > reference: https://twitter.com/ClaudeDevs/status/2097369738968195513/ > date created: 2026-09-08 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Reducing cost and improving performance with Claude Platform > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Reducing cost and improving performance with Claude Platform — Claude Blog](https://claude.com/blog/reducing-cost-and-improving-performance-with-claude-platform) (the canonical article; the thread is a repost) - [claude-api skill on GitHub](https://github.com/anthropics/skills/tree/main/skills/claude-api) - [Optimizing for cost and intelligence](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence) - [Prompt caching docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) · [TTL support](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#ttl-support) · [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching) - [Cache diagnostics](https://platform.claude.com/docs/en/build-with-claude/cache-diagnostics) - [Effort](https://platform.claude.com/docs/en/build-with-claude/effort) · [changing effort mid-conversation](https://platform.claude.com/docs/en/build-with-claude/effort#changing-effort-mid-conversation) - [Tool use with prompt caching — `defer_loading`](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-use-with-prompt-caching#defer-loading) - [Compaction](https://platform.claude.com/docs/en/build-with-claude/compaction) · [mid-conversation system messages](https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages) - [Batch processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing) · [Usage and cost API](https://platform.claude.com/docs/en/manage-claude/usage-cost-api) - [Cost optimization cookbook](https://platform.claude.com/cookbook/cost-optimization-cost-optimization#prompt-caching) - [Lessons from building Claude Code: prompt caching is everything](https://claude.com/blog/lessons-from-building-claude-code-prompt-caching-is-everything) - [Cognition: Devin fusion](https://cognition.com/blog/devin-fusion) (cited for the "switch model when the cache is already broken" point) - Link check performed against Reader's stored markup: 24 `<a href>` elements present. This source is densely cited; the citations are almost entirely to the vendor's own documentation. ## 📝 Summary - **What happened.** Four Anthropic engineers published a cost-reduction guide covering three levers — prompt cache hit rate, prompt anti-patterns, and effort calibration — and shipped three Claude Code commands that automate them: `/claude-api prompt-audit`, `/claude-api cost-optimize`, and `/claude-api hillclimb`. - **Why it matters.** The headline claim inverts the usual assumption: on their benchmarks, removing prompt scaffolding cut cost **14.6%** while *raising* accuracy **5.3%**. If that generalizes, the prompt-engineering habits accumulated over three model generations are now a tax rather than an insurance policy. - **Key stakeholders.** Teams running Claude in production at volume; anyone maintaining a large CLAUDE.md, skill library, or agent tool set; agent framework authors whose scaffolding is now the thing being audited. - **Risks and tradeoffs.** Every number is vendor-run, self-reported, and measured on prompts the same team constructed. The anti-pattern experiment *planted* the flaws it then removed, so it measures the tool's ability to undo known damage, not the prevalence or cost of that damage in real prompts. The held-out test set is 14 tickets. One reported figure disagrees with its own heading (73% vs 72%). And on SWE-bench, part of the saving came from constraining output length — a changed deliverable, not a free lunch. - **Bottom line.** The mechanisms are sound and mostly checkable against your own bills: cache prefix stability, effort calibration, and dead scaffolding are real cost centers. Treat the specific percentages as vendor marketing and the *levers* as the takeaway — then measure your own. ## 📖 Main Content The article's organizing claim is that cost and performance are not a fixed trade-off for most applications, and that three specific fixes recover both at once. **Prompt caching** is presented mechanically rather than as a feature. Before generating, Claude processes the prompt into an internal working state — prefill — which is the expensive part of handling input. Caching stores that key-value state so a request beginning with the same prefix reads it back instead of recomputing, billed at a fraction of full input price. Three constraints follow directly from that mechanism and explain nearly every cache miss: the cache is pinned to a specific model, reads must be byte-exact across the prefix, and entries have a limited TTL. The practical failures the authors list are all corollaries. A dynamic timestamp or ID in the system prompt breaks byte-exactness. Tool definitions that reorder themselves break it too, because the messages API renders tools at the top of the assembled prompt. Forked conversations — subagents, branches — only inherit the parent's cache when the fork's prefix is byte-identical, on the same model, at the same effort. And effort settings themselves render into the prompt ahead of the content, so changing effort mid-conversation is a cache break on most models; only select models including Opus 5 and Fable 5.1 allow it without one. The remedies are equally concrete: monitor hit rate through Claude Console and the cache diagnostics API, which reports miss reasons and the exact divergence point between two requests; mark rarely used tools `defer_loading` so they stay out of the cached prefix and are appended only when tool search pulls them in; add system instructions as mid-conversation messages rather than editing the system prompt; order the request so static context sits first and the growing conversation behind it; make model or effort changes at moments when the cache is already broken, such as compaction; move the breakpoint as the conversation grows using automatic caching; pre-warm with a `max_tokens: 0` request at session start; and switch to a 1-hour TTL when tool calls or subagents can outrun the 5-minute window, which counts from request start rather than from the last activity. **Instructions** is the section with the counterintuitive result. The argument is that prompts accumulate patches for weaknesses that no longer exist, and those patches now actively cost money on frontier models. Six named anti-patterns: verification rituals ("verify twice"), thoroughness and emphasis boosters ("be maximally thorough," "CRITICAL: YOU MUST ALWAYS"), mandatory procedures and scratchpad scaffolds, stale few-shot examples tuned to an older model's failures, contradictory rules, and dated configuration such as manual thinking budgets. The mechanism for each is specific rather than hand-waved: better instruction-following means a frontier model takes literal instructions literally, so a verification ritual becomes a duplicated tool call on every request and an emphasis booster becomes dozens of unnecessary searches. The supporting experiment is where care is required. The team started from a clean prompt on a customer support benchmark, planted one anti-pattern at a time to produce six legacy prompts, then ran each on Opus 4.8, on Opus 5 with only the model ID changed, and on Opus 5 after one `prompt-audit` pass. Averaged across the six, the audit cut cost 14.6% and raised accuracy 5.3%. The accuracy gain decomposes into three named failures, which is the most credible part of the section because each is a distinct mechanism rather than a statistical wash: a retired thinking setting made the API reject every routing request outright; contradictory refund rules led the model to withhold four refunds it owed while asking the customer to confirm; and a manual scratchpad collided with built-in thinking, so on three tickets the model wrote a tool call inside its reasoning and never executed it. That last failure mode is worth remembering independently of the benchmark. **Effort** is framed as how hard the model works, and the article's contribution is showing that the cost-performance curve differs by model and task rather than assuming higher is better. On FrontierCode Diamond, Fable 5 goes from 11.5% at low effort ($5.35/task) to 30.9% at max ($19.00/task) — roughly 2.7x the score for 3.5x the cost, a case where effort clearly pays. On Humanity's Last Exam without tools, Fable 5.1 goes from about 53% at low (~$0.30/question) to about 61% at max (~$2.23), with the final step buying about half a point for 46% more cost — a gain the authors say falls inside run-to-run noise. Miscalibration runs both directions: too high produces over-thinking, added latency, and sometimes degraded answers, because deliberation only helps while there is still evidence to find; too low produces answers built on partial information that nonetheless look finished, because the model stops making tool calls and skips checks it would otherwise run. The calibration advice is the sharpest practical claim in the article: test stronger models at *lower* effort. On CursorBench 3.2, Fable 5.1 at low effort matches Fable 5 at high effort for a third of the cost, and the authors decompose why — less work per task, plus cache reads priced at $0.25 per million tokens against $1.00 for Fable 5. They note that even at Fable 5's prices, Fable 5.1 at low effort would still cost about 40% less, which separates the pricing change from the capability change instead of letting the discount carry the claim. **Automation** closes the loop. `hillclimb` runs the search: given an evaluation split into train and test, it proposes configuration changes and reads failing train examples to fix what it finds. Their run started from Opus 4.8 at default high effort, moved to Opus 5 at low effort with anti-patterns removed (98.9% train accuracy, 2.6 cents per ticket), stepped down to Sonnet 5 at low effort (1 cent per ticket but 88.9% accuracy), then recovered Sonnet 5 to 98.9% at the same cost by adding routing rules and a refund-cap cross-reference derived from the failing tickets. On 14 held-out tickets the final configuration scored 90.5% against the original's 78.6% at about one-fifth the cost. `cost-optimize` runs the broader audit — profiling spend from the Admin API, from logged `usage` objects, or by reading request-building code — then ranks savings across caching, request trimming, output bounding, and batching. Four public benchmarks with Sonnet 5 as baseline: LegalBench ~58% lower cost with thinking tokens falling from 102,779 to 8,284; tau2-bench retail ~73% (the body text says 72%); OfficeQA Pro ~52%, $136.20 to $64.87; SWE-bench Verified ~55%, with median steps per task dropping 29 to 17 and prompt tokens 75.2M to 33.7M. ## 🗒️ Notes - Prefill is named as the expensive part of input handling; caching stores the KV state and cache reads bill at a fraction of full input price. - Three cache constraints, each the direct cause of a class of misses: model-pinned, byte-exact prefix, limited TTL. - Effort settings render into the prompt *ahead of* content, which is why changing effort mid-conversation normally breaks the cache. Opus 5 and Fable 5.1 are named exceptions. - Volatile values (timestamps, IDs) in the system prompt are a cache killer — the most common self-inflicted miss. - Tool definitions render at the top in a fixed order via the messages API; any change to a definition invalidates the cache. - Subagent and branch forks share the parent cache only when prefix, model, and effort all match exactly. Directly relevant to any multi-agent setup. - `defer_loading` keeps rarely used tools out of the cached prefix; they arrive via tool search only when needed. - The 5-minute TTL counts **from the start of the request**, not from last activity — a subagent or tool call running past 5 minutes expires the parent's cache before its result returns. 1-hour TTL is the documented remedy. - Pre-warming: `max_tokens: 0` with an explicit breakpoint and the same effort as real traffic, fired while the user is typing. - Compaction already rewrites much of the cache, making it the cheap moment to switch model or effort. - Six anti-patterns, all now cost centers on frontier models: verification rituals, emphasis boosters, mandatory procedures and scratchpads, stale few-shot examples, contradictory rules, dated configuration. - Headline result: **-14.6% cost, +5.3% accuracy** on average across six prompts, each carrying one deliberately planted anti-pattern. - Three named accuracy failures, all mechanism-level: a retired thinking setting caused outright API rejection; contradictory refund rules caused four wrongly withheld refunds; the manual scratchpad collided with native thinking so three tool calls were written inside reasoning and never executed. - FrontierCode Diamond (hardest 50), Fable 5: 11.5% @ $5.35/task low → 30.9% @ $19.00/task max. ~2.7x score for ~3.5x cost. - HLE without tools, Fable 5.1: ~53% @ ~$0.30 low → ~61% @ ~$2.23 max; last step ≈ +0.5 pt for +46% cost, inside run-to-run noise. - CursorBench 3.2: Fable 5.1 low effort ≈ Fable 5 high effort at one third the cost. Decomposed: less work per task + cache reads at $0.25/M vs $1.00/M; ~40% cheaper even at the old prices. - Over-thinking is named as a real failure, not a hedge: "Deliberation only helps while there's still evidence to find." - Under-effort failure is described as answers that "look finished" while built on partial information — the harder one to detect. - Flat cost-performance curve across an effort sweep on a non-saturated eval ⇒ the task is not bound by thinking compute. That is a clean, usable diagnostic. - hillclimb trajectory: Opus 4.8 high → Opus 5 low + audit (98.9% train, 2.6¢/ticket) → Sonnet 5 low (1¢, 88.9%) → Sonnet 5 low + routing rules and refund-cap cross-reference (98.9%, 1¢). Held-out 14 tickets: 90.5% vs 78.6% at ~1/5 cost. - cost-optimize spend discovery has a three-tier fallback: Admin API cost reports → logged `usage` objects → reading request-building code and estimating. - LegalBench: thinking tokens 102,779 → 8,284, pass rate within noise, ~58% cheaper (shared prefix caching, low effort, Batch API). - tau2-bench retail: heading says ~73%, body says 72%. Same paragraph, two numbers. - OfficeQA Pro: $136.20 → $64.87 (52.4%, consistent with the stated ~52%) via batch processing and document caching. - SWE-bench Verified: ~55% cheaper via medium effort **and constraining output to a few concise sentences**; median steps 29 → 17, prompt tokens 75.2M → 33.7M. The output constraint changes the deliverable. - Byline appears only in the final line of the thread: Lance Martin (@RLanceMartin), Brad Abrams (@brada), Isabella He (@IsabellaKHe), Ben Lehrburger (@benlehrburger). Reader recorded the author as "ClaudeDevs," the posting account. ## 📚 References - [Claude Blog: Reducing cost and improving performance with Claude Platform](https://claude.com/blog/reducing-cost-and-improving-performance-with-claude-platform) — primary source; the thread is a repost of it. - [anthropics/skills — claude-api](https://github.com/anthropics/skills/tree/main/skills/claude-api) — the skill carrying `prompt-audit`, `cost-optimize`, and `hillclimb`. - [Prompt caching documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) - [Cache diagnostics](https://platform.claude.com/docs/en/build-with-claude/cache-diagnostics) - [Effort documentation](https://platform.claude.com/docs/en/build-with-claude/effort) - [Optimizing for cost and intelligence](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence) - [Batch processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing) - [Usage and cost API](https://platform.claude.com/docs/en/manage-claude/usage-cost-api) - [Lessons from building Claude Code: prompt caching is everything](https://claude.com/blog/lessons-from-building-claude-code-prompt-caching-is-everything) - [Cognition — Devin fusion](https://cognition.com/blog/devin-fusion) — the one substantive third-party citation. - Benchmarks named but not linked to independent leaderboards: FrontierCode Diamond, Humanity's Last Exam, CursorBench 3.2, LegalBench, tau2-bench retail, OfficeQA Pro, SWE-bench Verified. The customer support benchmark is internal and unpublished. ## 🗣️ Paraphrase - **Opening.** Tuning caching, instructions, and effort cuts Claude's cost without giving up application performance. - **Framing.** Cost and performance are usually treated as a trade-off; in practice three fixes recover both — maximize cache hit rate, strip anti-patterns when moving to a frontier model, and calibrate effort. The guidance now lives in the claude-api skill. - **Cache mechanism.** Prefill turns your prompt into working state, and that is the expensive part of input. Caching saves the KV state so a matching prefix is read back rather than recomputed, at a fraction of the price. - **Cache constraints.** Pinned to a model, byte-exact across the prefix, limited TTL. Everything else follows from those three. - **Cache pitfalls.** Effort changes mid-conversation, volatile values in the prefix, reordering tool definitions, and non-identical forks all break the cache. - **Cache remedies.** Watch hit rate and miss reasons; defer rare tools; add system updates as messages; keep static context first; change model or effort when the cache is already broken; move the breakpoint as the conversation grows; pre-warm at session start; raise the TTL when calls can run past five minutes. - **Anti-patterns.** Prompts accumulate patches for weaknesses frontier models no longer have, and those patches now burn tokens: verification rituals, emphasis boosters, mandatory procedures and scratchpads, stale examples, contradictory rules, dated configuration. - **The audit.** `/claude-api prompt-audit` scans prompts, skills, and tool descriptions in the working directory, including CLAUDE.md and application code. - **The experiment.** Six prompts, one planted anti-pattern each, run on Opus 4.8, on Opus 5 unchanged, and on Opus 5 after an audit. Average result: 14.6% cheaper, 5.3% more accurate. - **Why accuracy rose.** A retired thinking setting caused outright API rejection; contradictory rules withheld four owed refunds; the manual scratchpad swallowed three tool calls that never executed. - **Effort.** Low effort concludes faster; high effort deliberates, verifies, and explores. The curve's shape varies by model and task. - **Effort economics.** Fable 5 on FrontierCode Diamond pays for max effort; Fable 5.1 on HLE does not, since the last step buys half a point inside the noise for 46% more. - **Miscalibration.** Too high over-thinks and can degrade the answer; too low produces confident answers built on partial evidence. - **Calibration advice.** Try a stronger model at lower effort — Fable 5.1 low matches Fable 5 high on CursorBench 3.2 at a third of the cost. A flat curve across an effort sweep means the task is not thinking-bound. - **Hillclimb.** Given an evaluation, it splits train and test, proposes changes, and reads failing cases. Their run reached 90.5% on held-out tickets against 78.6% at about a fifth the cost. - **Cost-optimize.** Profiles where tokens go, ranks savings, and, with an evaluation, measures effort and model trade-offs; four public benchmarks came in 52–73% cheaper. - **Getting started.** prompt-audit after a model migration, cost-optimize for a spend audit, hillclimb when you have an evaluation and want a search. ## 📰 Newsletter **Your CLAUDE.md might be the expensive part.** Anthropic engineers published cost-reduction guidance this week with a result that lands awkwardly for anyone who has spent two years hardening prompts: stripping the scaffolding cut cost 14.6% *and* raised accuracy 5.3% on their benchmark. The named offenders will look familiar — "verify twice," "be maximally thorough," "CRITICAL: YOU MUST ALWAYS," mandatory step-by-step procedures, scratchpad templates, and few-shot examples tuned to a model that retired two generations ago. The mechanism is the interesting bit, not the percentage. Frontier models follow instructions more literally, so a verification ritual becomes a duplicated tool call on every single request. In their test the manual scratchpad collided with the model's own thinking and three tool calls got written *inside* the reasoning and never executed — a failure that looks like a model bug right up until you delete the scaffold. For those of us running agents on a budget, the caching section is the part with money in it. The 5-minute cache TTL counts from the **start** of the request, so any subagent or tool call that runs longer than five minutes expires the parent's cache before its result comes home — and forked conversations only inherit the cache when prefix, model, and effort match byte for byte. If you run subagents, that is probably where your bill is. Three commands ship with the [claude-api skill](https://github.com/anthropics/skills/tree/main/skills/claude-api): `prompt-audit` (scans your prompts, skills, and CLAUDE.md), `cost-optimize` (profiles spend), and `hillclimb` (searches model and effort settings against your eval). Fair warning on the numbers: every benchmark here is vendor-run, the anti-patterns were planted by the same team that measured the fix, the held-out test set is 14 tickets, and one figure disagrees with its own heading. The levers are real. Measure them on your own bill before quoting anyone's percentage. ## ✅ Action Items #### DevRel - Developer advocacy: a talk on "the scaffolding tax" — why prompt hygiene from 2024 costs money in 2026, using the planted-anti-pattern experiment as the worked example, with its limits stated. - Education: a workshop on reading cache diagnostics; most teams have never opened the miss-reason report. - Developer experience: publish a before/after CLAUDE.md diff showing which instructions an audit removes and what it costs to keep them. - Developer marketing: any cost claim we repeat gets our own measurement attached, not the vendor's percentage. - Community management: collect real audit results from practitioners — the in-the-wild prevalence number is the one the article does not have. - Internal/external: share cache-miss findings upstream; the diagnostics API is new enough that field reports are useful. #### OSPO - Policy & Governance: decide whether agent configuration files are reviewed artifacts. If CLAUDE.md changes model cost and behavior, it belongs under change control. - License Compliance: the claude-api skill is Anthropic-published — confirm license terms before vendoring it into internal tooling. - Security Management: `prompt-audit` reads everything in the working directory including application code; scope it deliberately in any repo holding secrets. - Software Inventory: track model IDs and effort settings as inventory. "Which model runs in production" is currently undocumented in most orgs. - Developer Support: office hours on effort calibration; the failure mode of too-low effort is invisible without an eval. - Community Engagement: ask whether an open, non-vendor benchmark of these levers exists. If not, that is a real gap. - Strategic Alignment: cost-per-task is a reportable KPI now that the tooling exists to measure it. #### Community - Maintainers: audit project agent instructions the same way you audit dependencies — stale scaffolding accumulates identically. - Contributors: good first issue — grep a skills library for emphasis boosters and contradictory rules and file what turns up. - Sponsors: fund an independent replication of the anti-pattern experiment on prompts nobody planted. - Ecosystem: coordinate a shared list of anti-patterns across agent frameworks so each tool is not maintaining its own. #### WordPress - Core: no direct action; this is API-side guidance. - Core Performance: the methodology transfers — sweep a parameter, plot cost against outcome, and look for the flat region. Same discipline as a performance budget. - Core AI: any WordPress AI feature calling a hosted model should log token usage per request; without it, the cost-audit path here is unavailable. - Core Program Team: if contributor tooling adopts agent configs, someone owns reviewing them. - Mobile: no direct action. - Playground: a Playground-hosted demo of an agent with and without scaffolding would make the argument testable in a browser. - Community: not applicable to this source. - Training: a Learn lesson on prompt hygiene would date quickly — teach the audit method, not the anti-pattern list. - Marketing: not applicable to this source. - Support: if support tooling adopts an LLM, the customer-support benchmark shape here is the closest published analogue. - Design: no direct action. - Documentation: if docs tooling uses an LLM, bounded output is the cheapest lever named here. - Photos: not applicable to this source. - TV: not applicable to this source. - Polyglots: translation workloads are batchable and cache-friendly — the LegalBench pattern (shared prefix, low effort, Batch API) maps directly. - Accessibility: the source itself fails 1.1.1 — seven of eight images carry `alt="Image"`. Worth citing as an example when arguing that alt text is not optional for data figures. - Meta: no direct action. - CLI: if wp-cli grows AI commands, effort and model selection should be flags, not constants. - Hosting: hosts reselling AI features should read the caching section before pricing them. - Tide: not applicable to this source. - Openverse: not applicable to this source. - Themes: not applicable to this source. - Plugins: plugins bundling LLM calls should log `usage` objects; without them a cost audit has to guess from code. #### The WPOCC - Programs: an independent, non-vendor cost benchmark is exactly the kind of public good a collective can fund and no vendor will. - Projects: a shared prompt-hygiene checklist for WordPress-adjacent AI tooling. - Funding experiments: microgrant for replicating the anti-pattern result on unplanted, real-world prompts. ### FAIR - Technical independence: cost optimization that only works through one vendor's tooling is a dependency. Note which levers (caching, batching, output bounding) are portable across providers and which are not. - Policies and covenants: token spend and model selection are operational facts users of a federated service may reasonably expect disclosed. - Productized governance artifacts: add "documents its model, effort, and cost per operation" to the scorecard for any AI-using service. ## 🧪 Hypotheses 1. Running `prompt-audit` against a real, organically grown CLAUDE.md and skill library produces a smaller cost reduction than 14.6%, because real prompts carry fewer anti-patterns per token than deliberately planted ones — and possibly a larger one, because they carry more than one at a time. 2. Subagent-heavy workflows lose more to the 5-minute TTL than to any other cache mechanism, measurable as the share of parent requests whose prefix is recomputed after a long child call. 3. For most production tasks, the cost-performance curve across effort is flat over at least two adjacent levels, meaning default effort is above the knee for a majority of applications. 4. Removing verification rituals raises accuracy only on models with native extended thinking; on models without it, the same removal degrades accuracy. 5. The "stronger model at lower effort" result holds across providers, not just within the Claude family — a weaker-model-working-hard configuration is generally dominated. 6. Output-length constraints account for a larger share of the SWE-bench saving than the effort change does, which would make that benchmark's headline number partly a scope reduction rather than an efficiency gain. ## 📊 Methodology - **Source methodology, as reported.** Anti-pattern study: one clean baseline prompt on an internal customer support benchmark, six variants with one planted anti-pattern each, three arms (Opus 4.8; Opus 5 with only the model ID changed; Opus 5 post-audit), results averaged across the six. Effort study: benchmark scores and per-task cost at low and max effort on FrontierCode Diamond, HLE (no tools), and CursorBench 3.2. Hillclimb: train/test split on a customer support benchmark, 14 held-out tickets. Cost-optimize: four public benchmarks against a Sonnet 5 baseline. - **Weaknesses to note before citing.** No confidence intervals except the qualitative "inside run-to-run noise" on HLE. No sample sizes for the anti-pattern runs. The held-out set is 14 items, where a single ticket moves accuracy about 7 points. The internal benchmark is unpublished, so the baseline cannot be inspected. All arms were run by the vendor whose product is the remedy. - **H1 test.** Take 20 real agent configurations from different teams, measure cost and task accuracy before and after `prompt-audit`, and report the distribution rather than the mean. Pre-register the eval so the audit cannot be tuned to it. - **H2 test.** Instrument an agent framework to log prefix recomputation against child-call duration; correlate cache-miss cost with calls exceeding 5 minutes. - **H3 test.** Sweep effort across all levels on 10 production task types with a fixed eval; report the curve, not the endpoints. A flat region is the finding. - **H4 test.** Same prompt with and without a verification instruction, across model families with and without native thinking. This is the cheapest study listed and would sharpen the guidance considerably. - **H5 test.** Cross-provider matrix of model tier against effort or reasoning setting at matched cost, scored on a shared eval. - **H6 test.** Ablate the SWE-bench configuration: effort change alone, output constraint alone, both. The article reports only the combination. ## ❓ Questions - How prevalent are these anti-patterns in prompts nobody planted them in — and what do they cost there? - Does `prompt-audit` distinguish scaffolding that patches a model weakness from scaffolding that encodes a business rule? The contradictory-refund-rules example sits uncomfortably close to that line. - If a manual scratchpad can swallow a tool call so it is written but never executed, how would an application detect that in production without an eval? - Which of these levers survive a provider switch, and which are Claude-specific lock-in? - Does the 5-minute TTL counting from request start mean deep agent hierarchies are structurally cache-hostile? - What does "accuracy stayed within noise" mean numerically on LegalBench — the phrase carries a 58% cost claim with no interval attached. - If frontier models no longer need mandatory procedures, what happens to skill libraries — including this one — built entirely out of mandatory procedures? ## 🔮 Future Research - Independent replication of the anti-pattern result on real, unmodified production prompts across several organizations. - A public, non-vendor benchmark for cost-per-task that treats price as a first-class metric alongside accuracy. - Characterization of the cache economics of agent topologies — how depth, fan-out, and child-call duration interact with TTL and prefix stability. - Study of instruction-following literalism as a scaling property: does the cost of redundant instructions rise monotonically with capability? - Investigation of the failure mode where a scratchpad captures a tool call inside reasoning: how common, which models, what detection signature. - Longitudinal study of prompt drift — how quickly a prompt written for one model generation becomes a net cost on the next. ## 💡 Implications - **Practice.** Treat agent configuration as code that ages. An instruction added to fix a 2024 failure is a liability by 2026, and nothing in a normal workflow ever prompts you to remove it. Add a re-audit step to every model migration. - **Policy.** Model, effort, and cost per operation are becoming reportable operational facts. Organizations running agents at volume without per-request usage logging cannot audit their own spend. - **Security.** `prompt-audit` reads the whole working directory, including application code. That is a sensible default for a cost tool and a poor one for a repo holding credentials — scope it. - **Education.** The teachable content here is the method (sweep, measure, look for the flat region), not the anti-pattern list, which will be stale within two model generations. Curricula that teach the list will teach the next generation of anti-patterns. - **Funding.** The missing artifact is an independent cost benchmark. Every number in circulation comes from the vendor selling the remedy, and that is a public-goods gap, not a conspiracy. ## 🧠 Learning Styles Assessment - Reading level estimate: **Flesch–Kincaid grade 10.2, Flesch Reading Ease 51.7** (computed on the extracted body text: 2,210 words, 132 sentences, 16.7 words per sentence). Dense with domain terminology (prefill, KV cache, TTL, breakpoint, effort) but with short sentences and mostly defined terms. #### 👀 Visual learners - Present: strong — seven figures carrying the quantitative evidence (cache miss reasons, request layout, cost/accuracy comparisons, effort curves, hillclimb trajectory, benchmark savings). - Improve: the figures are images without meaningful alt text, so their content exists only visually. Numbers in the figures should also appear in the prose or a table. #### 🔊 Aural learners - Present: none — no audio version or narration. - Improve: the three-lever structure (cache, instructions, effort) is a natural 10-minute podcast segment. #### 🗣️ Verbal learners - Present: strong. Terms are defined at first use — prefill, KV cache, effort, breakpoint — and the anti-patterns are named with quoted examples. - Improve: "effort" is a product term doing work as an ordinary word; a one-line note on how it maps to reasoning budget would prevent confusion. #### 🧩 Kinesthetic learners - Present: strong — three runnable commands, and every recommendation is something you can execute against your own code today. - Improve: a sample repo with a deliberately bad prompt to audit would let readers see the tool work before pointing it at production. #### 🧠 Logical learners - Present: strong. The cache section derives its recommendations from three stated constraints, which makes the advice checkable rather than memorizable. - Improve: no confidence intervals, no sample sizes, and one internally inconsistent figure (73% vs 72%) undercut an otherwise rigorous presentation. #### 🧑‍🤝‍🧑 Social learners - Present: posted as a thread on X, so replies are the discussion surface. - Improve: no forum, issue tracker, or discussion link for reporting real-world audit results. #### 🧘‍♀️ Solitary learners - Present: complete and self-contained; a reader can work through it alone with their own codebase. - Improve: a self-check — "run the effort sweep on one task and find your flat region" — would convert reading into practice. #### 🌿 Naturalistic learners - Present: none explicitly. - Improve: token reduction is energy reduction. LegalBench dropping from 102,779 to 8,284 thinking tokens is a sustainability story the article never tells. ### 📊 Accessibility Assessment #### Screen reader semantics - Present: proper `<h2>` structure for all five sections; ordered and unordered lists used correctly. - Improve: **verified failure** — seven of the eight `<img>` elements carry `alt="Image"`, and the eighth repeats the article title. Every figure holding quantitative evidence (Figures 1–7) is inaccessible to screen reader users. WCAG 2.2 AA **1.1.1 Non-text Content**. The fix is descriptive alt text plus the figure's key numbers in the prose. #### Keyboard navigation - Present: text and links only; no custom widgets, no traps. - Improve: platform-controlled. 2.1.1 Keyboard. #### Contrast and readability - Present: platform default themes meet 4.5:1. - Improve: contrast *inside* the chart images is not author-controlled at render time and cannot be inspected here. 1.4.3 Contrast (Minimum), 1.4.11 Non-text Contrast. #### Text resizing and reflow - Present: body text reflows at 200%. 1.4.10 Reflow. - Improve: bitmap charts do not reflow and become unreadable when scaled — SVG or a data table would fix both this and 1.1.1. #### Media and captions - Present: no audio or video, so no caption obligation. - Improve: `<figcaption>` count is zero. Figures are referenced as "Figure 1" in prose with no programmatic association. 1.3.1 Info and Relationships. #### Forms and interactions - Present: none in the content. Not applicable to this source. - Improve: n/a. #### Standards reference - Primary gap: **1.1.1 Non-text Content** (alt="Image" on seven data figures). Secondary: **1.3.1** (no figure captions or programmatic association) and **1.4.10** (raster charts do not reflow). The prose carries most key numbers, which partially mitigates 1.1.1 but does not satisfy it. ### 🌍 DEIB Assessment #### Cultural breadth - Present: limited by genre — the examples are English-language benchmarks and a US-style customer support workflow. - Improve: cost sensitivity is far higher outside well-funded US teams; the cheapest-configuration finding matters most to readers the article does not address. #### Inclusive language - Present: neutral and professional; no gendered or exclusionary phrasing. Jargon is defined at first use. - Improve: "frontier models" is unexplained marketing vocabulary standing in for a technical property. #### Neurodiversity support - Present: strong — consistent structure, a "How to fix it" block in every section, short paragraphs, bulleted recommendations. Highly predictable. - Improve: the numbers are scattered across prose; a single summary table would reduce working-memory load. #### Representation - Present: four authors credited by name and handle in the closing line. - Improve: the byline appears last and Reader recorded the author as the posting account. All examples are the vendor's own; no customer or third-party voice appears. #### Psychological safety - Present: notably good. The article tells readers their existing prompt engineering is now counterproductive without implying they were wrong to write it — the framing is drift over time, not incompetence. - Improve: naming that the guidance itself will age would extend the same generosity to the next reader. ### ✅ Logical Validity #### Strong reasoning - **Deductive, prompt cache section.** Three stated constraints (model-pinned, byte-exact, limited TTL) entail every listed failure and remedy. Strong because a reader can derive new cases the article does not list — the mark of a real mechanism rather than a tip list. - **Causal, instructions section.** The claim that better instruction-following turns redundant instructions into cost is supported by three distinct named mechanisms (API rejection, contradictory rules withholding four refunds, scratchpad swallowing three tool calls). Three different failure modes converging is stronger evidence than a single aggregate percentage. - **Inductive with honest limits, effort section.** Two contrasting curves are presented — one where max effort pays (FrontierCode Diamond) and one where it does not (HLE) — and the authors state that the HLE gain falls inside run-to-run noise. Reporting a null result they were not required to report raises the credibility of the section. - **Decomposition, CursorBench claim.** Separating the pricing change ($0.25/M vs $1.00/M) from the capability change, and stating the result would still hold at old prices (~40% cheaper), pre-empts the obvious objection instead of hiding behind the discount. #### Fallacies - **Constructed-evidence / begging the question** — the anti-pattern experiment. The team planted the six anti-patterns in a clean prompt, then measured their own tool removing them. This establishes that the audit can undo damage the authors chose and inserted; it does not establish that real prompts carry this damage or that removal yields 14.6% in the wild. *Fix:* report the same measurement on a sample of unmodified production prompts, or state the headline as an upper bound. - **Conflict of interest / appeal to self-interested authority** — the whole article. Every benchmark is run by the vendor selling both the models and the remedy, and the recommended action is adopting their tooling. Not disqualifying, but no independent replication is offered or invited. *Fix:* publish the harness and the internal benchmark so a third party can reproduce it. - **Internal inconsistency** — the tau2-bench retail bullet: the heading claims "~73% lower cost" while the body says "reduced spend by 72%," in the same sentence pair. Small, but it is the one place a reader can check the article against itself, and it fails. *Fix:* pick one number. - **Suppressed premise / equivocation on "performance"** — SWE-bench Verified. The savings came partly from "constraining the agent's output to just a few concise sentences." Shorter output is a changed deliverable, not preserved performance, yet it is reported under a heading about cost reduction with performance held constant. *Fix:* report pass rate alongside the output constraint, and state explicitly what the user no longer receives. - **Hasty generalization from a small sample** — the hillclimb result. 90.5% versus 78.6% on 14 held-out tickets means roughly one ticket is worth 7 points; the headline gap is under two tickets wide. *Fix:* report the interval, or enlarge the test set. - **Overgeneralized advice risk** — "mandatory procedures and scratchpad scaffolds" are named as anti-patterns without distinguishing scaffolding that patches a model weakness from scaffolding that encodes a required business process. A reader following this literally could strip a compliance step. *Fix:* state the distinction; the article's own contradictory-refund-rules example shows the authors are aware of it. #### Tone of voice - Technical, measured, first-person plural. Notably free of hype: it reports a null result, names its own tools' limits in passing, and explains mechanisms rather than asserting outcomes. The rhetorical move is generosity toward the reader's existing work ("prompts can accumulate instructions that patch model weaknesses") — drift, not error. The weakness is not tone but sourcing: confident numbers with no intervals, no sample sizes, and no independent check. ### 🛡️ DARVO Analysis - **Deny:** none. No accusation is made or answered; the article is technical guidance, not a defense. - **Attack:** none. The closest thing is criticizing prompting practices, and it is explicitly framed as capability drift over time rather than as reader error. - **Reverse victim/offender:** none present. Not applicable to this source. - **Emotional cues and rhetoric:** the persuasive move is reassurance rather than blame — a reader is told their working prompts are now costly without being told they were wrong. The relevant critical lens for this source is commercial interest, not DARVO: the vendor benchmarks its own remedy with no independent replication, which is a sourcing problem rather than a rhetorical manipulation. ## 🎯 Conclusion - **Bottom line.** Three levers, all real and all checkable against your own bill: keep the cached prefix byte-stable (and watch the 5-minute TTL against long subagent calls), delete scaffolding written for models that have since retired, and calibrate effort per task instead of defaulting high. The mechanisms hold up under scrutiny; the specific percentages are vendor-run and, in the anti-pattern case, measured against damage the vendor planted. - **Next steps for readers.** Open cache diagnostics and find your actual miss reasons before changing anything. Run one effort sweep on one real task and look for the flat region. Then read your own CLAUDE.md and skill library for verification rituals, emphasis boosters, and mandatory procedures — and when you remove them, measure, because the 14.6% was produced under conditions your prompts do not share. ## 🔄 Refactor Appendix — Reducing cost and improving performance with Claude Platform ### 📚 Literature Atoms (H3) #### [H1] Prompt cache TTL counts from request start, not last activity > [!quote] > The 5-minute cache TTL counts from the start of the request. If an agent blocks on tool calls or subagent requests that run longer than 5 minutes, the parent's cache expires **Context & use**: - Prompt cache section, final tip. - The single most actionable fact here for anyone running subagents. It means deep agent hierarchies are structurally cache-hostile unless the TTL is raised, and it explains bills that look inexplicable relative to request count. **Tags**: #xref <!-- atomicity: 1 idea ✓ --> #### [H2] Frontier models turn verification rituals into duplicated work > [!quote] > With Opus 5, verification rituals ("verify twice") use unnecessary tokens by duplicating order lookup on every refund. **Context & use**: - Instructions section, explaining why the audit raised accuracy while cutting cost. - The mechanism, not the percentage, is the transferable part: better instruction-following makes redundant instructions literally expensive. **Tags**: #xref #addendum <!-- atomicity: 1 idea ✓ --> #### [H3] Manual scratchpads can swallow tool calls entirely > [!quote] > the manual scratchpad collided with Opus 5's built-in thinking: on three tickets it wrote the tool call inside its reasoning and never executed it. **Context & use**: - Instructions section, third of three named accuracy failures. - A silent, hard-to-detect failure that looks like a model bug until the scaffold is removed. Worth remembering independently of everything else in the article. **Tags**: #xref #share <!-- atomicity: 1 idea ✓ --> #### [H4] A stronger model at low effort can beat a weaker model at high effort > [!quote] > on CursorBench 3.2, Claude Fable 5.1 at low effort matches the performance of Fable 5 at high effort at a third of the cost **Context & use**: - Effort section, the calibration advice. - Inverts the standard routing instinct of dropping to a cheaper model when costs rise. Directly contradicts a default in my own credit-routing policy and should be tested against it. **Tags**: #xref #articleseed <!-- atomicity: 1 idea ✓ --> #### [H5] A flat effort curve means the task is not thinking-bound > [!quote] > a flat cost-performance curve across effort levels suggests that the task is not bound by thinking compute; increasing effort is not beneficial. **Context & use**: - Effort section, "Understand your task shape." - A clean diagnostic that turns effort selection from guesswork into a measurement. The cheapest experiment in the whole article. **Tags**: #xref <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] Prompt scaffolding ages into a cost center **Claim** (one sentence, Courtney's voice): Instructions written to patch a model's weakness become a tax on the next model generation, because nothing in normal work ever prompts their removal. **Evidence pointer**: [[#H2]] **Scope & conditions**: Holds for capability-patching scaffolding — verification rituals, emphasis boosters, procedural templates. Does not hold for instructions encoding business rules, formatting contracts, or safety requirements, which are requirements rather than patches. The distinction is whether the instruction describes *what* is wanted or *how* to think. **Links**: - [[Configuration Drift]] — because extends - [[Business Rules vs Model Patches]] — because contrasts on what the instruction encodes - [[CLAUDE.md Maintenance]] — because applies-in my own quarterly prune <!-- atomicity: 1 idea ✓ --> #### [P2] Measure the curve before choosing the setting **Claim** (one sentence, Courtney's voice): Sweeping a parameter across its range and finding the flat region beats reasoning about which value should be right. **Evidence pointer**: [[#H5]] **Scope & conditions**: Holds wherever the sweep is cheap relative to the decision and an evaluation exists. Fails when no eval exists, in which case the sweep measures nothing — which is the actual blocker for most teams, not the sweep itself. **Links**: - [[Empirical Over Inferential]] — because extends - [[A finding is a claim until something runs]] — because evidence-for - [[Credit Routing Policy]] — because applies-in model and effort selection <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Audit my own CLAUDE.md and skills library for the six named anti-patterns. Immediate suspects: the superpowers `using-superpowers` skill ("ABSOLUTELY MUST," "This is not negotiable," "You cannot rationalize your way out of this") and the mandatory-procedure structure of several skills including this one. #fleeting #inbox - [F2] The deep-read pipeline runs long subagent and script calls — check whether the 5-minute TTL is expiring parent caches during batch runs, and whether a 1-hour TTL applies. #fleeting #inbox - [F3] Test H4 against my credit-routing policy: routing matrix says drop to a cheaper model, this says try the stronger model at lower effort. One of them is wrong for my workloads. #fleeting #inbox - [F4] tau2-bench figure disagrees with its own heading (73% vs 72%). Minor, but note it if I ever cite the number. #fleeting - [F5] Article seed: "Your prompt hygiene is now a line item" — the scaffolding-tax argument, with an honest account of why the vendor's 14.6% is an upper bound. #articleseed - [F6] Accessibility example for talks: seven data figures shipped with alt="Image" by a company whose product reads images. #share ### → Things 3 (max 2 actions) - [ ] Run /claude-api prompt-audit against ~/.claude (CLAUDE.md, skills, hooks) and record what it flags before changing anything — AI config maintenance #things - [ ] Check whether long subagent calls in the deep-read pipeline are expiring the 5-minute prompt cache, and test a 1-hour TTL — pipeline cost audit #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Tweets/Reducing Cost and Improving Performance With Claude Platform.md|Reducing Cost and Improving Performance With Claude Platform by Lance Martin, Brad Abrams, Isabella He, Ben Lehrburger]] ## Highlights - - **Don’t exceed the prompt cache TTL**. The 5-minute cache TTL counts from the start of the request. If an agent blocks on tool calls or subagent requests that run longer than 5 minutes, the parent's cache expires before the result comes back. In cases like this, consider [setting a 1-hour TTL](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence) on the prefix instead. ([View Highlight](https://read.readwise.io/read/01m21cmg9mgmpfed7r0h0scq28)) - Note: Most actionable fact here for subagent-heavy workflows: the TTL runs from request start, not last activity, so any child call over 5 minutes expires the parent's cache before its result returns. Directly relevant to the deep-read pipeline's own long script and subagent calls. - Tags: prompt-caching, ai-agents, atom/literature/h1 - For example, we tested a model migration from Opus 4.8 to Opus 5 on a customer support benchmark. We started from a clean prompt and planted one anti-pattern at a time (a retired thinking setting, a pair of contradictory refund rules, a manual scratchpad, "verify twice", "be maximally thorough", and a mandatory six-step procedure), giving six legacy prompts. ([View Highlight](https://read.readwise.io/read/01m21cmm73pgs7e0gdbfbxpmgv)) - Note: The methodological caveat behind the headline number. The team planted the anti-patterns it then measured its own tool removing. That establishes the audit can undo damage the authors chose and inserted — not that real prompts carry this damage, nor that removal yields 14.6% in the wild. Treat the headline as an upper bound. - Tags: fallacy/constructed-evidence, llm-cost-optimization, methodology - Running */claude-api prompt-audit* removed the anti-patterns, decreasing costs by 14.6% and increasing accuracy by 5.3% on average. Cost dropped because extra tool calls and duplicated reasoning were eliminated. Accuracy rose for three reasons. The retired thinking setting made the API reject every routing request outright. The contradictory refund rules led Opus 5 to withhold four refunds it owed while it asked the customer to confirm. And the manual scratchpad collided with Opus 5's built-in thinking: on three tickets it wrote the tool call inside its reasoning and never executed it. ([View Highlight](https://read.readwise.io/read/01m21cmra0kkvz1jvrz1pdqwd8)) - Note: Headline result plus the three named mechanisms. The scratchpad failure is the one worth remembering independently: scaffolding captured the tool call inside reasoning so it was written but never executed — a silent failure that looks like a model bug until the scaffold is deleted. - Tags: prompt-engineering, atom/literature/h2, llm-cost-optimization - **Test stronger models at lower effort**. A stronger model at low effort can be cheaper than a weaker model working hard (high effort). For example, on CursorBench 3.2, Claude Fable 5.1 at low effort [matches the performance](https://www-cdn.anthropic.com/0339e6a7c5c7b87f5c07798616dc32c215d14235/Claude%20Fable%205.1%20&%20Claude%20Mythos%205.1%20System%20Card.pdf) of Fable 5 at high effort at a third of the cost (Figure 5). Two things make the newer model cheaper: at low effort it does less work per task, and Fable 5.1's prompt-cache reads are priced at $0.25 per million tokens versus $1.00 for Fable 5. Even at Fable 5's prices, Fable 5.1 at low effort would cost about 40% less. ([View Highlight](https://read.readwise.io/read/01m21cmwpdsgtgnkzj1kf1q7r6)) - Note: Inverts the usual routing instinct of dropping to a cheaper model when cost rises. Contradicts a default in my own credit-routing policy — worth testing. Credit to the authors for decomposing the pricing change from the capability change instead of letting the discount carry the claim. - Tags: atom/literature/h4, llm-cost-optimization, credit-routing - - **Understand your task shape.** Measuring application performance across a [sweep of effort levels](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#tune-effort) is a useful way to understand the cost-performance tradeoff for your particular task. On a non-saturated evaluation, a flat cost-performance curve across effort levels suggests that the task is not bound by thinking compute; increasing effort is not beneficial. ([View Highlight](https://read.readwise.io/read/01m21cn0k8c16m73323pb82va7)) - Note: The cheapest experiment in the article and the cleanest diagnostic: sweep effort, look for the flat region, and stop paying for compute the task cannot use. Turns effort selection from argument into measurement — assuming you have an eval, which is the real blocker for most teams. - Tags: llm-cost-optimization, methodology, atom/literature/h5 - cost-optimize found that the default config already caches correctly. Savings came from setting effort to medium and constraining the agent’s output to just a few concise sentences. Median steps per task went from 29 to 17 and prompt tokens fell from 75.2M to 33.7M. ([View Highlight](https://read.readwise.io/read/01m21cn2yr4nag0q8csznwbs2c)) - Note: Reported as cost reduction with performance held constant, but part of the saving comes from constraining output to "a few concise sentences" — a changed deliverable, not preserved performance. Ablate the two levers separately before citing the ~55%. - Tags: fallacy/equivocation, llm-cost-optimization