AI infrastructure is enormous — data, storage, networking, serving. This series zooms into the one slice where the money and the scarcity live: compute. The whole series answers one question — are you getting useful work out of the silicon you pay for?

A simple model that splits a compute cluster into four layers — reservation, provisioning, scheduling, workload — so 'my pod is Pending' becomes a one-minute diagnosis instead of a three-team argument.

The most common — and most expensive — GPU cluster question. The same 'Pending' can come from any of the four layers; here's how to find which one, before you waste money on the wrong fix.

This post is only about one thing inside the accelerator: its ability to calculate. The same chip has a ~10× range of 'speed' depending on precision — but that number is a ceiling, and most of the time you never reach it.

The second thing inside the accelerator: its memory. Two numbers — capacity (does it fit) and bandwidth (can you feed the compute) — and which one limits you depends on the workload, not the chip. This is why the FLOPs ceiling from 1.1 is rarely your speed.

Why 8 GPUs in a box aren't just 8 GPUs — the high-bandwidth links inside a node and what they mean for tightly-coupled work.

How boxes talk — InfiniBand/RoCE/EFA and the rack/pod layout; where a job lands across the network changes its speed as much as whether it lands.

The last piece of a unit of compute isn't on the accelerator at all — it's the CPU, system RAM, and local disk feeding it across a thin PCIe straw. When the host starves the GPU, no faster GPU helps — and its fix isn't in this Part. This post closes the anatomy and bridges into the rest of the series.

A whole accelerator handed to a job that uses a sliver of it is the most common waste; the fork between splitting in space and sharing in time.

Splitting one GPU in space into isolated instances — strong isolation, coarse granularity, and when it fits.

Sharing one GPU concurrently (MPS) or by turns (time-slicing) — higher utilization, weaker isolation.

Sharing only pays off if one job can't steal another's memory or tail latency; where each sharing mode's isolation breaks.

The loop every placement decision runs through — framed as where each later problem actually occurs, not a docs tour.

A multi-GPU job needs all its pieces at once; partial placement deadlocks and wastes what it grabbed.

Free GPUs you can't use because they're scattered; packing tight vs spreading out, and what each costs.

Where a job's pieces land changes its speed as much as whether they land; keeping tightly-coupled work close on the fabric.

Guaranteeing each team a floor without letting the cluster sit idle behind that guarantee — lend the spare, take it back on demand.

Not enough to go around — who gets evicted, how, and why evicting too eagerly costs more than it saves.

Serving is provisioned for peak and idle most of the day; backfill it with batch without hurting latency.

Demand has a daily rhythm; ride it instead of provisioning for peak and paying for the trough.

Jobs that expand into idle GPUs and contract when demand returns — absorbing the spare instead of leaving it idle.

Saving progress cheaply so an interruption costs minutes, not days — and the tradeoff between checkpoint overhead and lost work.

Giving capacity back without losing work — spot reclaim, preemption, and the orderly cordon → drain → evict dance.

At scale, hardware fails and slows constantly; detecting the fail-slow node before it burns a whole run's cycles.

Three numbers people all call 'utilization' that disagree — and which one tells you you're wasting money.

A dollar figure on a single idle percentage point at fleet scale — the number that justifies the whole series.

Why a paid commitment must run hot to beat on-demand, and when spot wins.

How much to reserve is a prediction problem; sizing committed capacity against forecast demand instead of guessing.

The whole series in one argument — supply and scheduling aren't two problems, and the seam between them is the leverage.