> [!question] Lingering Questions > - What does maintaining the HNC fork actually cost in engineering time, and what's the plan if it diverges further from upstream? > - What utilization improvement did gang scheduling and bin-packing actually deliver? No figures are given anywhere in the talk. > - How many production Kubernetes platforms are carrying similar orphaned/archived dependencies without recognizing it as a counterexample to "just assemble the ecosystem"? > [!tip] Article Ideas > - "Does your quota system protect the control plane?" — short piece on the object-count failure mode: a tenant can stay inside every compute quota and still degrade the cluster by object count alone. > [!link] Cross Reference > - [[3. Resources/Permanent Notes/Fairness Must Protect the Control Plane as Well as Allocated Compute]] — already promoted from this exact source (2026-08-11); not re-promoted here > - [[3. Resources/Permanent Notes/Cheap Generation Raises the Total Maintenance Bill]] — a different mechanism (cheap generation) reaching the same conclusion (maintenance burden is inherited, not avoided) as this source's archived-dependency counterexample > - [[3. Resources/Permanent Notes/Validation That Inspects Steps Cannot See Composition]] — the layered-boundary argument (network + API + controller-watch-scope) is a defense-in-depth instance of the same "single inspection point can't see composed behavior" family > [!quote] Vocabulary > - **gang scheduling** — a job's pods all start together or not at all, to avoid accelerators sitting idle while waiting for the rest of a synchronous job > - **HNC (Hierarchical Namespace Controller)** — community project providing hierarchical tenant structure over Kubernetes' flat namespaces; archived upstream, forked and maintained in-house here > - **bin-packing** — placing workloads tightly to preserve contiguous free capacity, avoiding fragmentation that strands large pods despite adequate aggregate space ## Bib capsule Ten-minute KubeCon Japan 2026 talk describing how Preferred Networks built a multi-tenant AI platform on Kubernetes, sharing heterogeneous accelerators (GPUs and their own MN-Core chips) across internal and external researchers. ## Author's thesis Multi-tenancy here is an economic conclusion, not an architectural preference: accelerators dominate cost and cannot be left idle, so sharing is forced, and everything else — the layered boundary, the fairness regime, gang scheduling — exists to compensate for the isolation that sharing gives up. The talk frames this as enabled by "assembling" CNCF ecosystem components rather than building from scratch, though its own account discloses two counterexamples to that framing without recognizing them as such. ## Key arguments 1. **Multi-tenancy is forced by accelerator economics**, not chosen for architectural elegance — cost pressure, not design preference, drives the whole platform. 2. **A boundary must be layered because no single layer is sufficient** — network policy, admission control, and (the most easily missed) restricting what controllers are permitted to watch cluster-wide. 3. **Fairness must extend to the control plane**, not just compute — a tenant can exhaust the API server through object count alone while staying inside every resource quota. ## Claim atoms ### Fairness must protect the control plane as well as allocated compute Where/why: ~00:07:00 — already promoted to [[3. Resources/Permanent Notes/Fairness Must Protect the Control Plane as Well as Allocated Compute]]; not re-promoted here. ### Assembling from the ecosystem still leaves maintenance to inherit > [!quote] ID: rw_igarashi-multitenant-2026 > Although the original community project was archived, we currently maintain our own fork. Where/why: ~00:05:31 — the talk's own unrecognized counterexample to its "we no longer build from scratch" thesis; gang scheduling required a custom plugin for the same reason. New Permanent below. ### Partial allocation of a synchronous job is worse than none Where/why: ~00:07:32 — gang scheduling justified from mechanism: started pods hold expensive accelerators idle while waiting for the rest of the job to schedule. ## Permanent candidates - **Inheriting maintenance when adopting an ecosystem component** — promoted below. - Partial-allocation-is-worse-than-none for synchronous jobs — held; fairly specific to distributed-training scheduling, revisit if a second independent domain instance appears. ## Processing checklist - [x] Checked existing Permanents — found one already created from this source (2026-08-11) - [x] Created the missing Literature Note - [x] Created one new Permanent note for the archived-dependency counterexample - [x] Connected into relevant MOC ## Metadata - Source: [Building a Multi-Tenant AI Platform With the CNCF Ecosystem - Aya Igarashi, Preferred Networks, Inc.](3.%20Resources/Readwise/Full%20Document%20Contents/Articles/Building%20a%20Multi-Tenant%20AI%20Platform%20With%20the%20CNCF%20Ecosystem%20-%20Aya%20Igarashi,%20Preferred%20Networks,%20Inc..md) - is_nar_related: false ## Links - [[3. Resources/MOCs/Open Source Governance MOC]] — the archived-dependency-becomes-a-fork pattern is directly relevant to dependency-health governance