← Reading versionAN ALLOCATION STORY

Inside the workload-management project

Somewhere,
two GPUs
are waiting.

A request arrives. Follow the decisions that turn it into a place on the machine.

Enter the system A narrated visual model of the project, not a live cluster.
A request for two GPUsScroll through the story to follow the request. The prose describes each decision.REQUEST / JOB 422 × GPUThe work is yours. The placement is ours.A MESSAGE IN TRANSIT01001010 00101010 00000010APITransport carries intent. Admission gives it context.Server / admissionCLUSTER A1 GPU unitToo smallCLUSTER B4 GPU unitsSmallest fitCLUSTER C16 GPU unitsAlso fitsIllustrative capacities · topology is configured per clusterCONFIGURATION → TOPOLOGY → ELIGIBLE NODE VIEW1D ring2D mesh2D torus3D torusDashed green links show representative wrap connections.PRIORITY ORDER / NEXT ATTEMPT →Job 57waitingJob 48waitingJob 422 GPUs needed1 free / 2 requiredNo placement attempt can manufacture capacity.ONE ROW OF A MESH · NO WRAPNode 1171(0, 0)occupiedNode 1287(1, 0)distance 1Node 1391(2, 0)distance 2Node 1495(3, 0)distance 3Three free units. Two will become an allocation.Spine 1Spine 2Leaf 1Leaf 2Node 12GPU 87 freeNode 13GPU 91 freeNode 14GPU 95 freeNIC downA cable can exist without a usable path.Job 42 / 2 GPUsAllocationOWNERSHIP LINK 1GPU 87assigned to Job 42OWNERSHIP LINK 2GPU 91assigned to Job 42A quantity becomes two exact resource identities.

01 / The request

The computation is still a mystery.

A user asks for two GPUs. Perhaps they will train a model. Perhaps they will simulate a universe. The allocator does not need to know the experiment. It needs a request it can reason about.

Job 42 · GPU × 2 · illustrative request

02 / The crossing

Intent becomes a message.

Across the network, the request is just data. At the API boundary, it becomes a job: an identity, a resource requirement, and an owner whose quota must be checked.

The travelling bits are a visual metaphor for an API request.

03 / The gateway

Which cluster gets the question?

The coordinating Server checks admission and chooses the smallest cluster whose total resource-unit count fits the request. That gives the job a destination. It does not guarantee that two GPUs are free right now.

Cluster selection and exact resource selection answer different questions.

04 / The shape

A cluster carries its own geometry.

A ring closes a line. A mesh connects neighbours. A torus wraps its edges; a third dimension adds another direction. The cluster defines the dimensions and wrap rules. Its nodes provide coordinates. Placement uses the resulting view instead of hard-coding one machine shape.

Selected links are shown here. Our worked request uses a four-node, non-wrapping mesh row.

05 / The queue

At the front. Still waiting.

The queue decides whose turn it is. FIFO follows arrival; priority ordering follows job priority. Our request needs two GPUs, but only one is free. Under the default capacity policy, it waits—and this queue pass stops here.

Quotas govern admission. Queue order governs the next attempt.

06 / The opening

A completion changes the picture.

An earlier job completes. GPUs 91 and 95 are released. The queue is checked again. Now three GPUs are free for a request that needs two. Counting has done its part. Placement begins.

Node 11 remains occupied. It will give the placement policy a reference point.

07 / The placer

Look through the machine.

The Placer sees current, locked inventory. Each cluster supplies its topology configuration; the eligible nodes become a fresh topology view. Coordinates become distances. A list of machines becomes a space in which a policy can make a choice.

Try both policies. Same inventory. Same request. A different preference.Explore the placement lens ↑

08 / Another lens

Free is not the same as reachable.

A coordinate layout describes geometry. A leaf–spine fabric describes paths through switches, ports, links, and node NICs. Its candidate query can reject a free GPU when its node has no usable path. Here, Node 14 has a down NIC.

The fabric query is a separate view of candidate discovery; it is not called by the coordinate Placer shown above.

09 / The assignment

Two units. One accountable decision.

The request said “two GPUs.” The allocation now names the exact units. Separate ownership links connect Job 42 to each selected GPU. The transaction commits that assignment. RUNNING records allocation here; executing the computation belongs to another layer.

The system can now answer not just “how many?”, but “which ones belong to this job?”

10 / The return

Ownership ends. Reality still matters.

When the job completes, the system follows those exact ownership links to release its resources. A healthy node’s GPU becomes available. A GPU on a DOWN node becomes unavailable. Finishing a job cannot repair a machine.

The story ends where the next request begins: with trustworthy inventory.

The architecture behind the journey

A boundary at
every decision.

Admission chooses a destination. The queue chooses a turn. Placement chooses units. Ownership makes the choice accountable.

Read the architecture story ↗Explore the source ↗
Follow the request again ↑