HyperAIHyperAI

Command Palette

Search for a command to run...

Online Tutorial丨Only About 3B Activated Each Time, MiniCPM-V 4.7 Achieves on-device Decoding Speed of 13+ tokens/s

Featured Image

If language models taught AI to read and express itself, then multimodal models further expand the ways it perceives the world. From recognizing objects in images and reading data in charts to understanding events that occur continuously in videos, the model no longer needs to process only text, but combinations of different information forms, and transform them into content that can be reasoned about and answered.

MiniCPM-V 4.7, launched by ModelBest and the OpenBMB community, is a multimodal vision-language model designed for visual understanding tasks. It adopts a 35B-A3B Mixture-of-Experts (MoE) architecture, with total parameters reaching 35B, but only about 3B parameters activated per token, effectively controlling inference computation overhead while retaining the capabilities of a larger model.

In visual understanding, MiniCPM-V 4.7 supports single-image, multi-image, and video understanding, and can handle different types of visual input. The model introduces Canvas M-RoPE visual positional encoding and efficient visual compression technology to help the model better capture spatial relationships and detailed information in images. When faced with complex visual content, it can not only recognize objects in the scene, but also understand image information in context and complete corresponding question-answering and analysis tasks.

More interestingly, such a model with 35B total parameters does not necessarily need to rely on large servers to run. MiniCPM-V 4.7 has been specially optimized for on-device inference: through technologies such as Lookahead Expert Scheduling (LES), it improves the expert invocation efficiency of the MoE model and achieves smooth on-device inference without sacrificing model performance; combined with fine-grained hierarchical quantization, DSpark speculative sampling, and other technologies, it further reduces runtime overhead. In the official demonstration, the model can even achieve a decoding speed of 13+ tokens/s with about 3GB of memory on an iPhone 17 Pro Max. In other words, multimodal models do not necessarily have to stay in the cloud; you can also experience visual understanding capabilities by picking up a phone.

HyperAI has now launched "MiniCPM-V 4.7: Multimodal Vision-Language Large Model (35B-A3B MoE)", allowing users to experience with a low barrier how the model understands visual inputs such as images, and further understand how the Mixture-of-Experts architecture balances model scale, computational overhead, and actual runtime efficiency through expert scheduling and inference optimization.

Run Online:

https://go.hyper.ai/w8kGy

More Online Tutorials:

https://hyper.ai/notebooks

demo page

Running the Demo

  1. After entering the hyper.ai homepage, select the "Tutorials" page, or click "View More Tutorials", select "MiniCPM-V 4.7: Multimodal Vision-Language Large Model (35B-A3B MoE)", and click "Run This Tutorial".
  1. After the page redirects, click "Clone" in the upper right corner to clone the tutorial into your own container.

Note: The upper right corner of the page supports language switching. Currently, Chinese and English are provided. This tutorial article uses English as an example for the steps.

  1. Select "2 x NVIDIA GeForce RTX 5090 (32 GB)" and the corresponding image, then click "Continue job execution".
  1. Wait for resources to be allocated. When the status changes to "Running", click "Open Workspace" to enter Jupyter Workspace.

Results Showcase

  1. After the page redirects, click the README file on the left. After entering, click Run at the top.
  1. After the run is complete, click the API address on the right to access the demo.