Tencent's Hunyuan Hy4 open-sourced: 770 billion total parameters, 1 million context window

Beating AI News: Tencent has released the Hy4 preview, a new Mixture of Experts (MoE) architecture model with 770 billion total parameters, 49 billion activated parameters per inference, and a context window expanded to 1 million tokens. Compared to Hy3’s 295B total parameters, 21B activated parameters, and 256K context window, this generation’s scale has more than tripled, while the context window is roughly four times larger. This time, Tencent did not solely emphasize general benchmark scores, but specifically enhanced the model’s performance on real-world tasks including coding, office work, gaming, and scientific research. For example, the model can build a Three.js 3D website from scratch, create playable game demos in Unity, and process up to 72 financial documents at once to check for duplicate reimbursements, over-limit expenses, and budget anomalies. Hy4 was also trained and iterated alongside Tencent products such as WorkBuddy and CodeBuddy. A standout feature is that Hy4 has begun participating in its own R&D: Tencent says the model helps optimize training methods, data strategies, evaluation systems, and underlying operators, proposing its own plans, running experiments, and adjusting based on results. The code, logs, and feedback from these experiments are fed into the next R&D cycle, forming an initial self-improvement loop. The Hy4 preview is now integrated with WorkBuddy, CodeBuddy, Yuanbao, ima, Tencent Cloud TokenHub, and OpenRouter. Tencent previously teased in its financial reports that Hy4 would be larger than Hy3, and noted real product feedback as a key source for model training.