Back to Research
Independent research
Speculative decoding on an A100 — what actually got faster
The headline is 1.39×. The finding worth reading is that the other method I tested was three times slower.
Results
Every value links to the committed artifact that produces it.
| Metric | Value | Evidence |
|---|---|---|
| Aggregate latency speedup — vLLM + EAGLE-3, Qwen3-8B, A100 80 GB | 1.387× | paper/data/phase3_results.csv |
| Peak on math-reasoning promptsone category; a latency speedup, not throughput | up to 1.46× latency | extracted_metrics.json:147 |
| Global throughput | 110.93 tok/s | extracted_metrics.json:146 |
| Study 1 (Transformers assisted decoding)a negative result, reported as "Study 1: no acceleration" | 0.25–0.36× — ~3× slower | paper/data/phase3_results.csv |
What didn’t work
Transformers-assisted decoding came out ~3× slower than the baseline. The README says so outright; the study reports the confound rather than hiding it.
Reported as a null, not hidden — the same honesty policy applies to every number on this site.
Running it
Live decoding needs 80 GB of VRAM; the site shows a dashboard built from the committed extracted_metrics.json plus the notebook as read-only HTML.