Tools
Coming soon
overshoot-roofline
A profiler for a running vLLM or SGLang server. It compares each part of a step with its hardware limit and shows how long the GPU waits on the CPU.
Fig. 1Example output
GPU time per step in ms, measured against the hardware limit, Qwen3-30B-A3B on one H100
MoE kernels5.42 / 4.011.41
Dense GEMMs1.40 / 0.740.66
Attention0.98 / 0.250.73
Norms and elementwise0.51 / 0.010.50
GPU waiting on the CPU0.80 / –0.80
Whole step9.2 / 5.04.2
Decode, Qwen3-30B-A3B, H100, 8 requests in flight, 9.2 ms per step against a 5.0 ms limit.
MoE kernels64.2 / 25.438.8
Dense GEMMs17.0 / 13.33.7
Attention9.0 / 2.76.3
Norms and elementwise12.8 / 5.96.9
GPU waiting on the CPU20 / –20
Whole step125.0 / 47.477.6
Prefill, Qwen3-30B-A3B, H100, 7 new requests and 1 decoding, 6,929 tokens, 125 ms per step against a 47.4 ms limit.