# A Removable Model Is the Practical Test of System Independence ## Claim A product's real independence from any single AI model is tested by whether its evaluations hold up when that model is removed and swapped for another — not by how many models it nominally supports. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Applies to AI products architected to route across multiple models, tools, and context sources rather than calling a single model directly. - "Removable" means the product's evaluation suite keeps passing at comparable quality, not merely that the code compiles with a different API endpoint plugged in. - Does not establish that any specific product currently passes this test — it's a design/audit criterion, not a report of a verified outcome. ## Evidence Satya Nadella frames AI product architecture as optimizing a cost-to-outcome frontier across models, context, tools, skills, memory, and harness design, and proposes a concrete falsifiable test for whether a product actually has control over that architecture: pull out any single model and see whether the product's own evaluations keep improving. The post reports this practice inside Microsoft products but supplies no benchmark evidence for the claim. ## Links - [[3. Resources/Permanent Notes/Tool-Interface Choice Shapes Agent Architecture More Than Model Choice]] — extends: both claims locate real product control in the harness/interface layer rather than the model itself; this note supplies a specific test for that claim. - [[3. Resources/Permanent Notes/Route Judgment to the Expensive Model and Bulk Execution to the Cheap One]] — applies-in: a routing strategy that varies model by task is exactly the kind of system this removability test is meant to audit. - [[3. Resources/Permanent Notes/A Capability Transfer Is Proven by Absence Not by a Handoff]] — because extends: the identical test applied to a person in an organisation rather than a model in a stack — removal, not declared support or a signed handoff, is the only evidence neither can fake. - [[3. Resources/MOCs/AI Agent Safety MOC]] — because it supplies the operational test behind every claim of model independence in this MOC: remove the model and see whether the evaluations still hold. ## Tests / falsifiers - Apply the test directly to a system I control (e.g., the deep-read pipeline or Librarium): swap the underlying model and rerun the existing evaluation set — does quality hold? - Find a product that claims model-independence but fails this test when actually tried, which would show the claim is more often asserted than achieved. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Readwise/Tweets/Frontier Diffusion & Control]] — the cost-to-outcome frontier framing and the removable-model independence test.