Transformer Accelerator Simulator
A cycle-accurate CPU and accelerator simulator for GPT-style forward passes, including systolic-array matrix multiplication, attention execution, SRAM/HBM transfers, and configurable tiling. Roofline and trace-driven analysis exposed the shift from compute-bound to memory-bound execution beyond 2K-token sequences.
At what context length does moving transformer state cost more than computing on it?
- HBM traffic at 8K
- 3.2× reduction
- Throughput at 8K
- 1.8×
- A100 estimate error
- ≤12%