Every chatbot reply is cooked in a GPU kitchen. Drag the sliders, flip the switches, and break things on purpose. The field guide and Kiely's Inference Engineering in about 90 minutes of play, with no reading required.
A GPU is a kitchen. Before anything else, meet the cast. Every term with a wavy red underline is tappable: hover or tap it for the plain-English meaning and its kitchen equivalent.
Covers Field guide §0–§1 · Kiely ch.1–2 · the same ground as your "How a Call Runs" one-pager, but moving
The recipe books (weights) must fit on the counter (HBM) before a single diner is seated. Whatever space is left holds order tickets (KV cache), one per diner, and each grows with the conversation's length. Drag the diners up until the counter overflows.
Covers Field guide §9(a)(b), §4.1 · Kiely ch.3, ch.5 · GPU specs from your Hyperscale Ledger
Each decode step hauls every recipe book across the kitchen once. Serve one diner per trip and the stove sits idle. Serve a whole table on the same trip and each extra plate is nearly free. Turn the dial and watch your speed against the kitchen's total.
Covers Field guide §0, §3.2 · Kiely ch.2 (arithmetic intensity), ch.5 · roofline cross-checked against DeepMind's How to Scale Your Model
A fast sous-chef (a small draft model) guesses the next k words. The head chef checks all of them in one trip and keeps the ones it agrees with, up to the first miss, plus one of its own. One diner gets several tokens per trip. That's different from batching, which is many diners per trip.
Now run a real dinner service. Requests arrive all minute: mostly short, a few huge. The inference engine is the head chef deciding, every few milliseconds, who is on the counter and who gets the next trip. Flip one switch at a time and watch who waits.
Covers Field guide §1–§3, §5 · Kiely ch.4–5 · the core idea: an inference engine is a scheduler for GPU memory
When the recipe books don't fit on one counter, you split them across several kitchens. There are three ways to cut them, and each costs a different amount of shouting between kitchens. Inside one building the hallway is fast (NVLink). Between buildings it's much slower (InfiniBand).
Covers Field guide §4 · Kiely ch.5 (parallelism) · cross-check: DeepMind scaling book, "Sharding" and "Inference" chapters
Your AI Stack atlas has the personal version of this (the Token Meter: should you pay per token or buy a Mac?). This is the company version: should a company rent its own GPU fleet, and what does it take to run one?
Covers Field guide §6, §8, §9(c) · Kiely ch.1, ch.7 · Darden GBUS 8496 Class 10 (Tryon, "Own the Weights")
Tryon labels financial sentences. Owning the tuned 1.7B model costs $19.23 per million labels plus the people to run it. Renting the API costs $890 (frontier), $356 (standard) or $35.60 (budget) per million. Your class answer was "owning wins only against the tier nobody retested." Drag the volume and see where that flips.
Meta's Llama 3 run saw 419 unexpected interruptions in 54 days on 16,000 GPUs: about one failure per 50,000 GPU-hours (Kiely ch.7).
How long until a new replica serves its first token? Shrink each bar.
Steps 0–5 all happen in one kind of kitchen: a datacenter GPU with its HBM counter. The same recipe books also get cooked in a food truck (the NPU in your laptop or phone), on a prep line where every book is already open on the counter (the LPU), and in a robot's onboard computer. Before any of that, they get written in a banquet hall the size of a warehouse. Three questions sort every one of them: where do the recipe books sit, how fast can they be hauled, and how much power does the kitchen draw?
Covers Chip guide §4, §6, §6a, §13 · Physical AI guide §2.4 · Llama 3 paper §3.3
For each kitchen, pick where its recipe books live. SRAM is on the chip itself: the fastest memory there is, but it holds only megabytes. HBM sits in the package right beside the chip. LPDDR is low-power memory on the board, shared by everything on the chip.
Training is one enormous dinner service in lockstep. Every table cooks its share of the course, then all of them compare notes (the all-reduce) before anyone starts the next one. So the hall runs at the speed of its slowest table (a straggler), holds far more on the counter than serving ever does, and has to be one building. The work is counted in FLOPs; how much of the stoves' time is real cooking is the MFU.
Three clients, three briefs. Pick the hardware, the model, the precision and the engine features, then submit. The plan is scored on whether it fits, meets the SLO, and stays in budget, using the same math as steps 1–5. A plan is some number of replicas, identical copies of the model, each split over 1–8 GPUs inside one machine, with users shared evenly across them. Rents: A100 $1.60 · H100 $2.50 · H200 $3.50 · B200 $6.00 per GPU-hour.
Configure, then submit.
Eight questions. The answers aren't on this page; you get them by operating the widgets in steps 1–5. Graded instantly, with rounding tolerance.
Exact: the memory arithmetic (weights, KV cache per token, users that fit), the MoE expert-coverage formula (assuming uniform routing; real routers are skewed), the speculative-decoding formula, the break-even math, and your Class 10 numbers. Teaching models: decode speed assumes 70% of peak bandwidth and compute; the scheduler sandbox uses round-number costs (prefill ≈ 25K tok/s; a decode step ≈ 12 ms plus a little per diner); the parallelism panel approximates communication with ring all-reduce and all-to-all volumes. The directions are the real ones; the exact milliseconds are not a benchmark. The capstone's GPU rents ($1.60–$6.00 an hour) are illustrative round numbers in the range of 2026 on-demand prices, not quotes. GPU specs come from the audited Hyperscale Ledger. GB means 10⁹ bytes throughout.