Prime Intellect Inference Platform Goes Live: Internally Processes Nearly 1 Trillion Tokens Daily

Beating AI Express News: Prime Intellect has launched an open-source model inference service called Prime Inference. Users can call models directly on demand; for tasks requiring long-term stable operation, they can reserve a portion of inference computing power in advance. The service’s interface is compatible with OpenAI API, and the first publicly deployed model is GLM-5.3. The system was previously used internally by Prime Intellect. The company states it processes nearly 1 trillion tokens daily, for use cases including reinforcement learning, synthetic data, model evaluation, and coding agents. Prime Intellect has also been running large-scale deployments for clients since January this year. Prime Inference focuses on optimizing long-context workloads for agents. It splits prompt processing and response generation across separate GPUs, cutting p90 inter-token latency by nearly 40% in tests. Prime Intellect also compressed KV cache, boosting the number of tokens each decoder can cache from 1.09 million to 1.63 million under the same GPU memory – an improvement of roughly 50%. Prime Intellect previously offered training infrastructure such as reinforcement learning, evaluation, and sandboxes. With the addition of the inference service, it can integrate model training and actual deployment into a single platform, while data generated from real-world operations can also be used for subsequent training.