Agentic Does Not Mean Latency-Sensitive Human-in-the-Loop Does

# Agentic Does Not Mean Latency-Sensitive, Human-in-the-Loop Does ## Claim Whether an agent workload needs low latency depends on whether a human is synchronously waiting on each turn, not on whether the workload is technically agentic or multi-turn.  ## Scope & conditions - Holds for resource-allocation and infrastructure decisions (concurrency limits, GPU pool sizing, routing priority) made about agent workloads specifically. - The common shorthand "agentic workloads are latency-sensitive" is true often enough to feel like a rule because many current agent products are chat-style and human-facing — but the property that actually matters is synchronous human waiting, and it's separable from the agentic label. - An autonomous, long-running agent task (no human watching each turn) should optimize for throughput even though it is fully agentic; a human-in-the-loop single-shot classification task should optimize for latency even though it is not multi-turn. ## Evidence LY Corporation's production platform: the top two use cases by volume (text summarization for agent memory, information extraction) are agentic/multi-turn but latency-insensitive, because no human waits synchronously on each turn. Their real-time, latency-sensitive use case (safety classification) is single-shot, not agentic at all. Infrastructure provisioning follows the human-waiting property, not the agentic label: "for human in the loop task where human interact often with agent latency is very important but for autonomous agent that runs a long task latency may be less important and throughput may become more important." ## Links - [[3. Resources/MOCs/AI Agent Safety MOC]] — applies-in; oversight posture and resource allocation for agent systems both depend on this same human-waiting distinction, not on the agentic/non-agentic label - [[3. Resources/MOCs/AI Cost & Tokenomics MOC]] — applies-in; GPU/inference cost optimization decisions should be scoped by this property rather than by workload category alone ## Tests / falsifiers - Audit a set of agent-system infrastructure decisions (concurrency caps, routing priority, provisioning) made under the "agentic = latency-sensitive" shorthand; check whether any autonomous, no-human-waiting workloads were over-provisioned for latency at the expense of throughput they didn't need. - Find an agentic workload where synchronous human waiting is absent but latency still matters for a different reason (e.g., a downstream system with its own SLA) — that would mean human-in-the-loop is not the only determinant, and the claim needs a stated exception. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Literature Notes/How To Evolve Your LLM Self-Hosting Platform]] — S. Omura & Y. Zhan, LY Corporation, CNCF/KubeCon talk