# Local Inference Is a Procurement Spectrum Not a Binary ## Claim You evaluate AI tooling along a spectrum where self-hosted open-weight models occupy the middle ground between paid frontier licenses and no AI at all. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Holds when routine tasks dominate the workload and privacy, latency, or predictable cost outweigh peak model capability. - Breaks down when a task genuinely requires frontier-level reasoning that only a hosted, closed model currently delivers. - Doesn't establish which local model is "good enough" for which task — only that the spectrum exists and is being actively used by real engineering teams. ## Evidence A LeadDev piece reports engineering managers adopting self-hosted local LLMs as "a third option emerging for managers to pick from," with the llama.cpp maintainer publicly attesting a 27B local model is "a very capable local model for coding tasks," and at least one adopter reframing the choice as technological sovereignty — continuity against a vendor withdrawing access — rather than only cost. ## Links - [[3. Resources/Permanent Notes/Local Models Are Sovereignty Insurance Against AI Access Cutoffs]] — because extends; that note states the general sovereignty claim, this one supplies the engineering-management adoption evidence for it. - [[3. Resources/Permanent Notes/Government-Imposed Access Restriction Is a Distinct AI Vendor Risk Category]] — because applies-in; the sovereignty framing here is a private-sector instance of that broader risk category. - [[3. Resources/MOCs/AI Cost & Tokenomics MOC]] — because applies-in; this is a live procurement decision framework, not just a technical curiosity. ## Tests / falsifiers - Run a routine coding-task suite through a local 27B-class model and a frontier hosted model, and compare accuracy, latency, and cost — if local inference underperforms badly on routine (not edge-case) tasks, the "viable third path" claim weakens. - Track whether adoption stays at pilot scale or moves into production workloads over the next few quarters — stalled pilots would suggest the spectrum framing is aspirational rather than operational. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Readwise/Full Document Contents/Articles/Engineering Managers Ditch Cloud AI for Local LLMs]] — reports the third-path framing and the llama.cpp attestation.