EAGLE-4 boosts Qwen’s generation speed by 3.6 times, delivering a 3x speedup even on mobile CPUs.

Beating AI News Flash: Large model inference acceleration firm Eagle has launched EAGLE-4, an upgrade to its speculative decoding solution. The new version focuses on cutting the computational overhead of the draft model, and adjusts prediction scale based on hardware performance and concurrency to avoid extra prediction calculations that would otherwise slow down speed. On 4 NVIDIA B300 GPUs, Qwen3.8-27B’s single-request generation speed jumped from 240.5 tokens per second to 863.8 tokens, 3.6 times faster than standard inference. When running 512 concurrent requests, total throughput rose from 21,200 tokens per second to 28,300, a 34% improvement. Significant gains were also seen on mobile: On the Motorola Razr 70 Ultra, Gemma-4-E2B-it achieved 3.2x faster long text generation speed solely on CPU, cutting the same task time from 43 seconds to 14 seconds, with 53% lower energy consumption per token. EAGLE-4’s code is now open-source, supporting integration with SGLang, vLLM, and experimental TokenSpeed. However, unlike the first three generations which used the Apache 2.0 open-source license, the new version adopts a restrictive commercial license. Enterprises with annual comprehensive revenue exceeding $20 million, and companies offering competing inference services, require separate authorization for commercial use.