Baidu ERNIE-5.5-Flash-Base

Overview

ERNIE-5.5-Flash-Base is a base model Baidu open-sourced on November 5, 2026: a sparse MoE with 280B total parameters and 14B activated per token, with weights stopped at the end of pretraining and no mid- or post-training applied. That is what sets it apart from most open releases. What you get is a clean starting point, and every decision about instruction tuning, continued domain pretraining or reinforcement learning stays with you, including the ability to compare different post-training recipes on the same checkpoint.

Pretraining consumed 33.5 trillion tokens with a 128K training context, while the released fine-tuning and inference configuration opens up to 1M, supported by both SGLang and vLLM at 1,048,576 tokens. The stack is 43 layers: one dense FFN layer plus 42 Multi-Router Latent MoE layers, each routing to the top 10 of 512 experts plus one shared expert, with a hidden dimension of 4,096 and a vocabulary of 201,216. Weights ship in both FP8 and BF16 on Hugging Face and ModelScope.

Against base models of a similar scale it leads on most benchmarks: MMLU-Pro 78.4 against 69.2 for DeepSeek-V4-Flash-Base, 66.9 for Hy3-preview-Base and 65.7 for MiMo-V2.5-Base, plus HumanEval+ 82.3, LiveCodeBench 40.1, LongBench-v2 48.7 and RepoQA 91.7. A few rows tilt the other way, such as BBH chain-of-thought at 89.2 against 90.0 for Hy3 and ChineseSimpleQA at 73.0 against 75.3 for DeepSeek-V4. The overall curve is broad rather than spiky.

Key Features

• 280B total, 14B active sparse MoE: 43 layers: one dense FFN plus 42 Multi-Router Latent MoE layers, each routing to the top 10 of 512 experts plus a shared expert, hidden dimension 4,096, vocabulary 201,216.
• Pretraining-endpoint weights only: No mid-training, no post-training. Baidu positions it as a starting point for domain adaptation, instruction tuning and reinforcement learning research on one shared checkpoint.
• 128K training context, 1M to fine-tune and serve: Pretraining topped out at 128K, while the released fine-tuning and inference configuration reaches 1,048,576 tokens, already supported by SGLang and vLLM.
• FP8 and BF16 in parallel: The main release is FP8 mixed precision with a separate BF16 variant, both on Hugging Face and ModelScope, so self-hosting can pick by VRAM and framework.
• Leading most same-scale base benchmarks: MMLU-Pro 78.4, HumanEval+ 82.3, LiveCodeBench 40.1, RepoQA 91.7 and LongBench-v2 48.7, with the clearest gaps on code and long context.
• Chinese and multilingual on par: CMMLU 90.0, ChineseSimpleQA 73.0, MMMLU-lite 81.0 and MultiLoKo 43.3, so Chinese capability was not thinned out to build a general base.

Use Cases

• Continued domain pretraining: finance, medical or legal corpora can keep training from this checkpoint instead of starting from zero.
• Post-training research: run SFT, DPO and RL side by side on identical base weights so results are not confounded by the foundation model.
• Long-context RAG: a 1M inference window removes much of the chunking and retrieval engineering, and RepoQA at 91.7 shows the repository-level grounding is there.
• Self-hosted inference: FP8 weights on SGLang or vLLM, with the deployment tier chosen by available VRAM.
• Chinese business foundation: CMMLU and ChineseSimpleQA put it in range as a fine-tuning base for Chinese products.

Pros

• The release is the pretraining endpoint, so neither researchers nor product teams have to unwind someone else's tuned preferences first.
• 280B total with 14B active keeps per-step compute and VRAM tied to the activation budget, lowering the self-hosting bar compared with dense models.
• A 128K training context with 1M for fine-tuning and inference lets long documents and repository-scale code go in whole.
• FP8 and BF16 weights ship together on both Hugging Face and ModelScope, which is friendlier for networks in mainland China.
• It leads same-scale base models on MMLU-Pro, HumanEval+, LiveCodeBench and RepoQA, giving solid code and long-context foundations.
• Chinese benchmarks show no weak spot, with CMMLU 90.0 and ChineseSimpleQA 73.0 at a level usable for Chinese fine-tuning.

Pricing

The release is a weight download, not a per-token service. FP8 and BF16 weights are free on Hugging Face and ModelScope, while commercial use and redistribution follow the ERNIE 5.5 Flash Base Model License Agreement v1.0, which differs from common permissive open-source licenses and is worth reading before shipping. Self-hosting costs depend on GPUs, precision and serving framework; Baidu published no hosted API pricing with the weights.

Summary

ERNIE-5.5-Flash-Base opens a 280B sparse MoE at the pretraining endpoint, with a 33.5 trillion token foundation, a 1M inference window and FP8 plus BF16 weights released together. Its value is not out-of-the-box usability but a clean starting point where post-training data and method stay in your hands. Teams wanting a finished chat model should wait for the official post-trained version, and the license terms deserve a read first.

Version History

• ERNIE-5.5-Flash-Base open-sourced at 280B total and 14B active (2026-11-05): Baidu open-sourced the base model ERNIE-5.5-Flash-Base on November 5, 2026, releasing the pretraining endpoint weights of ERNIE 5.5 Flash with no mid- or post-training. It is a sparse MoE with 280B total parameters and 14B activated per token across 43 layers: one dense FFN plus 42 Multi-Router Latent MoE layers, each routing to the top 10 of 512 experts plus a shared expert, hidden dimension 4,096, vocabulary 201,216. Pretraining used 33.5 trillion tokens with a 128K context, while fine-tuning and inference support up to 1,048,576 tokens on both SGLang and vLLM. Weights ship in FP8 and BF16 on Hugging Face and ModelScope, with commercial use under the ERNIE 5.5 Flash Base Model License Agreement v1.0. Against same-scale base models it leads with MMLU-Pro 78.4, HumanEval+ 82.3, LiveCodeBench 40.1, RepoQA 91.7 and LongBench-v2 48.7, and Baidu says a post-trained version will follow.