Overview
The limit for on-device multimodal models is not parameter count, it is memory. Even at aggressive 4-bit quantization a 35-billion-parameter model needs around 15 GB of weights, which no flagship phone can hold in working memory at once. MiniCPM-V 4.7 from ModelBest answers with a mixture of experts: 35 billion total parameters, 3 billion active per token, with most expert weights left in flash and read in on demand.
The cost shows up immediately. Every token passes through many layers, so the compute unit waits repeatedly for weights to arrive. The figure the team gave in an interview is 3 tokens per second on an iPhone 17 Pro Max running that way, which no user would sit through.
The layer added on top is lookahead expert scheduling, LES for short. While one layer is still computing, the inference engine predicts which experts the next layer will need and prefetches them. If the loading time hides entirely inside compute time, the read cost disappears. After tuning, the same iPhone 17 Pro Max went from 3 to 13 tokens per second at 3 GB of memory, and the README records a 5x speedup on real devices without giving up capability. DSpark speculative decoding heads arrived on Oct 15 for both the 1B and the 35B-A3B, in BF16 and GGUF.
The 1B version goes a different way. It does not chase flagship limits; the target is the budget handset. An inference researcher on the team said the 1B build reaches all the way down to thousand-yuan phones, and the previous 4.6 generation already hit roughly 15 tokens per second on some of them. The 1B weights are Apache-2.0 as well, with a full set of quantizations: BF16, int4, AWQ, GPTQ and GGUF, where int4 uses bitsandbytes and AWQ and GPTQ are both W4A16.
Inference frameworks cover Transformers, vLLM, SGLang, Ollama and llama.cpp, with an official openbmb/minicpm-v4.7 entry on Ollama. Operating systems cover iOS, Android and HarmonyOS; chips cover Intel, AMD, Arm, Rockchip and Houmo. The combination means a phone, a development board and a PC all work without fresh porting, starting from whichever quantization is already packaged.
The README lists image, multi-image, document and video understanding as the core capabilities. Audio is out of scope for this V model, which is the job of MiniCPM-o 4.5, so the two lines stay clearly separated. Context length is not stated in the README; third-party registries list 262,144 tokens, a figure with no official source behind it, so do not treat it as settled in a selection document.
In the past two months it has turned up on three quite different devices. An official video shows it running vision and robot planning locally on a Jetson AGX Orin; the community developer Rebel-Gloria uses the 1B as the default on-device vision model in PRTS, an iPhone and iPad app for blind and low-vision users; and ModelBest says the family already powers Galaxy AI on the Samsung Galaxy Z Fold8 and Fold8 Ultra. By November the MiniCPM family had passed 70 million cumulative downloads.
Two claims need separating. Co-chief scientist Yao Yuan said this generation's 35B model has passed the 2023 cloud flagship GPT-4V, which is a judgement from an interview rather than a published leaderboard score, and no test set was given. The 3 GB memory figure and 13 tokens per second come from the team's own run on an iPhone 17 Pro Max, documented in the README but without independent reproduction.
Three kinds of user fit. Anyone doing image, document or video understanding where there is no network; anyone squeezing vision into robots and development boards with tight compute; and anyone who wants weights they can download under a permissive licence to prototype with. The 35B-A3B suits flagships and Orin-class hardware, the 1B suits budget phones and smaller boards, and the memory budget decides between them.
Key Features
- Mixture of experts living in flash: 35 billion total parameters with 3 billion active, most expert weights left in flash and read on demand, trading read cost for memory footprint.
- Lookahead expert scheduling: The engine predicts which experts the next layer needs while the current one computes and prefetches them, reported as a 5x speedup on real devices.
- DSpark speculative decoding: Dedicated heads for both the 1B and the 35B-A3B in BF16 and GGUF, available since Oct 15.
- Every format, every runtime: Six quantizations across BF16, FP8, int4, AWQ, GPTQ and GGUF, and five runtimes across Transformers, vLLM, SGLang, Ollama and llama.cpp.
- Multimodal input: Single images, multiple images, documents and video, with text output; audio belongs to MiniCPM-o 4.5 instead.
- Permissive licence: Code and weights both ship under Apache-2.0, downloadable from Hugging Face and ModelScope.
Use Cases
- Offline image and document understanding on a phone
- Local vision and planning on robots and development boards
- Scene description inside accessibility apps
- On-device inspection and gauge reading in industrial settings
- Understanding local video where there is no network
Pros
- A 35-billion-parameter model at 3 GB of memory, the first usable speed at this size on device
- The 1B version reaches budget handsets, lowering the hardware floor
- Apache-2.0 weights with broad quantization and runtime coverage
- Three real deployments already exist across PRTS, Jetson and Samsung Galaxy AI
Pricing
Code and weights are open sourced under Apache-2.0 and downloadable from Hugging Face and ModelScope, with no licensing fee for self-hosting. ModelBest publishes no price for a hosted API, and third-party hosting routes bill on their own terms.
Summary
Memory bandwidth has always been the ceiling for on-device multimodal work, and MiniCPM-V 4.7 routes around it with a mixture of experts plus lookahead scheduling. A 35-billion-parameter model at 3 GB and 13 tokens per second is the first combination that holds up on this class of device, and the 1B brings the floor down to budget phones. Context length is unpublished and independent reproduction is still missing.
Version History
- MiniCPM-V 4.7 released (2026-10-12): ModelBest open sourced MiniCPM-V 4.7 in 35B-A3B and 1B versions. The 35B-A3B is the first mixture-of-experts model in the MiniCPM-V line, with 35 billion total and 3 billion active parameters, reaching 13 tokens per second at 3 GB of memory on an iPhone 17 Pro Max through lookahead expert scheduling, which the team reports as a 5x speedup on real devices. The 1B targets budget handsets. Both ship under Apache-2.0 with BF16, FP8, int4, AWQ, GPTQ and GGUF quantizations across Transformers, vLLM, SGLang, Ollama and llama.cpp, on iOS, Android and HarmonyOS.