Overview
Qwen3.8-Flash-Next is a multimodal MoE model open-sourced by Alibaba Tongyi Qianwen late at night on August 26, 2026, Beijing time. As a preview version of the next-generation Qwen4 architecture, it releases architectural changes ahead of time for global developers to test. The main model has a total of 125B parameters, with only 6B activated per token, and is equipped with an N-gram embedding layer of approximately 51B parameters. It natively supports a 256K context, expandable to 1M, and includes a built-in vision encoder. Its training cost is nearly 90% lower than Qwen3.7-Plus, and the weights have been open-sourced on Hugging Face and ModelScope, with SGLang providing Day-0 support.
Key Features
- Hybrid Attention Mechanism: Combines GatedDeltaNet linear attention with QwenSparseAttention sparse attention to optimize efficiency and quality for long sequences.
- N-gram Embedding: Stores common phrases as independent entries in a phrase dictionary, which can be placed in system memory rather than on the GPU, reducing VRAM usage.
- Gated Residual and Multi-Hyper-Connection Flow: Optimizes gradient flow and information propagation in deep networks, improving training stability and performance.
- Muon Optimizer: A next-generation training optimization scheme that significantly reduces training costs, to about one-ninth of Qwen3.7-Plus.
- Efficient Inference Architecture: Only 6B parameters are activated per token, and the official claim is that it can run locally on a single RTX 4090 without quantization, lowering deployment barriers.
- Ultra-Long Context Support: Native 256K context, expandable to 1M using YaRN technology, meeting large-scale document processing needs.
Use Cases
- Local low-latency inference: Runs on a single RTX 4090, suitable for individual developers or small to medium teams building private services.
- Long document understanding and analysis: Handles large codebases, academic papers, or legal contracts using 256K to 1M context.
- Multimodal tasks: Built-in vision encoder supports scenarios with image input and text joint reasoning.
- Coding assistance: Performs well on benchmarks like SWE-bench Pro and LiveCodeBench v6, suitable for complex programming tasks.
- Architecture research and validation: As a Qwen4 precursor model, allows developers to test new architectural features and inference stack compatibility.
Pros
- Training cost is nearly 90% lower than Qwen3.7-Plus, offering significant economic advantages.
- Only 6B parameters are activated during inference, enabling operation on consumer-grade hardware.
- Native 256K context, expandable to 1M, providing strong long-text capabilities.
- Multimodal MoE design balances performance and efficiency.
- Outperforms or is comparable to contemporary efficiency-focused models in multiple inference and coding evaluations.
- Weights are fully open-sourced, with an FP8 quantized version also provided, facilitating ecosystem integration.
Pricing
The sibling model Qwen3.8-Flash (non-Next version) is available on QwenCloud/Qianwen platform as an API service, priced at 1 yuan per million input tokens and 3 yuan per million output tokens (approximately $0.16/$0.47). Qwen3.8-Flash-Next is an open-source weight model, and specific usage costs should be referenced from the official website.
Summary
Qwen3.8-Flash-Next is an open-source precursor model by Alibaba Tongyi Qianwen paving the way for the Qwen4 architecture. With a highly efficient MoE design of 125B total parameters and 6B activated, combined with innovations like hybrid attention and N-gram embedding, it achieves low training costs and local deployment. It performs impressively on coding and inference benchmarks, and through open-sourcing, it promotes community ecosystem adaptation, laying the foundation for the full Qwen4 release.
Version History
- Qwen3.8-Flash-Next open-sourced (Qwen4 architecture preview) (2026-08-26): Alibaba Qwen open-sourced Qwen3.8-Flash-Next late at night Beijing time, a working preview of the next-generation Qwen4 architecture: a multimodal MoE with 125B total params, only ~6B active per token, plus a 51B N-gram embedding layer. Native 262K context extends to 1M via YaRN. Four architectural innovations (GatedDeltaNet + sparse-attention hybrid, N-gram embeddings, gated residual connections, Muon optimizer) preview Qwen4. Training cost is about 90% lower than Qwen3.7-Plus; an FP8 quantized version shipped the same day and weights are available on Hugging Face and ModelScope.