Where the bytes go
Each component, sized to its share of the per-GPU footprint.
| component | GB | % |
|---|---|---|
| Total | — | 100% |
Issue 01 · A small calculator for large models
How much memory does my
FSDP training run actually need?
A quiet, browser-based estimator for PyTorch FSDP / FSDP2 training. Pick a model, set the sharding and precision, read the answer. Source on GitHub.
Per-GPU peak memory
— GB
—
—
Calculation details
Step-by-step formulas behind each memory component.
Scaling with world size
Per-GPU peak as you add more workers, holding everything else equal.