# Aggregate Throughput Hides the Per-Stream Speed a User Feels ## Claim When a performance number omits its concurrency, I cannot tell whether it describes what the system produces or what any one user experiences, and vendors report whichever is larger. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Holds wherever a rate is summed across parallel workers but reported in the same unit a single worker would use — tokens/second, requests/second, pages/hour, words/minute. - The failure is not fabrication. Both numbers are true measurements of the same run; they differ only in denominator, and the denominator is the part left out. - Sharpest when the aggregate is stated immediately after language about individual capability, because the reader carries the individual frame into the number. - Breaks when concurrency is one, or when the audience genuinely cares about system output rather than personal latency — a batch pipeline's operator wants the aggregate, and reporting per-stream there would be the misleading choice. ## Evidence NVIDIA's DGX Spark walkthrough demonstrates sixteen concurrent Open Code sessions against one vLLM deployment and reports the result as "generating in total 500 plus tokens per second worth of work." Divided across the sixteen streams, each session sees roughly 31 tokens/second. The figure lands one sentence after the model is framed as "a workhorse model to complete a lot of simple tasks very quickly," so the reader arrives at the number already thinking about a single task's speed. Note also the non-standard unit: "tokens per second **worth of work**" is not a measurement anyone can reproduce, and the qualifier absorbs the ambiguity the missing denominator creates. ## Links - [[3. Resources/Permanent Notes/A Benchmark Measures Capability Not Lived Experience]] — contrasts on mechanism: that note is about a benchmark measuring the *wrong surface* (load test vs. dashboard), while this one is about the *right* surface measured with a suppressed denominator. Same skepticism, different failure — a number can be on-surface and still not describe your experience. - [[3. Resources/Permanent Notes/A Delivery-Throughput Metric Cannot See Whether Knowledge Spread]] — extends: both are throughput metrics that count output while staying blind to the thing the output was for. - [[3. Resources/Permanent Notes/A System Can Pass Every Tracked Metric and Still Fail Its User]] — evidence-for: a concrete instance of the tracked metric and the user's experience diverging while nothing is technically misreported. - [[3. Resources/MOCs/AI Cost & Tokenomics MOC]] — applies-in: local-inference cost comparisons are built on tokens/second figures, so the denominator question governs whether any of those comparisons mean anything. ## Tests / falsifiers - Re-run the same deployment at concurrency 1 and read the per-stream rate. If it lands near 500 tok/s rather than near 31, the aggregate reading is wrong. - Look for vendor demos that report both figures side by side. If reporting both is common, the "vendors report the flattering one" half of the claim is too strong and should narrow to "readers should ask" rather than "vendors choose." - Check whether the same pattern appears in units outside AI — CI minutes, orders/second, support tickets closed. If it does not generalize, this is a claim about AI marketing, not about metrics. ## Maintenance - Last tended: 2026-08-31