The four cost classes
Every operation is one of four cost classes, and the surface form names the class. You can read the cost of a program straight from the source, without running it:
| Cost class | Surface marker |
|---|---|
| Pure host op | let, var, for, a call to a non-GPU function |
| Upload to device | gpu let, gpu var (paid lazily at first capture) |
| Kernel launch | forall over device data, a gpu fn launch, .reduce |
| Fence + readback | Cross-residency assignment (let h = g), .slice(a..b) |
The runtime exposes counters so you can confirm the cost class of each line. After
gpu_reset_telemetry(), the functions gpu_uploads(), gpu_launches(),
gpu_readbacks(), and gpu_fences() return the cumulative counts. The
buffer-reuse recipe below asserts them directly.
Two properties keep repeated launches cheap: a device buffer is persistent — uploaded once,
reused by every subsequent kernel until the binding leaves scope (it is freed at scope exit) — and a
gpu let b = g move transfers the device handle without copying.