Instructions to use AXERA-TECH/MiniCPM-V-4.6-AX637 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AXERA-TECH/MiniCPM-V-4.6-AX637")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AXERA-TECH/MiniCPM-V-4.6-AX637", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AXERA-TECH/MiniCPM-V-4.6-AX637"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AXERA-TECH/MiniCPM-V-4.6-AX637

SGLang

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AXERA-TECH/MiniCPM-V-4.6-AX637" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AXERA-TECH/MiniCPM-V-4.6-AX637" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with Docker Model Runner:
```
docker model run hf.co/AXERA-TECH/MiniCPM-V-4.6-AX637
```

MiniCPM-V-4.6 on AXERA NPU

Ready-to-run deployment package for openbmb/MiniCPM-V-4.6 on AX637 / NPU1.

Runtime: packaged axllm with OpenAI-compatible server APIs.
Text runtime: rebuilt non-GPTQ W8A16 LLM artifacts.
Vision runtime: fixed-shape 448x448 MiniCPM-V-4.6 vision encoder.
Modalities: text chat, image understanding, and video understanding.
Context layout: prefill_len=128, prefill_max_token_num=1152, kv_cache_len=1535.

Supported Platform

AX637 / NPU1

Validated Devices

AX637 development board

Runtime Configuration

The default config.json enables mmap loading for embeddings and layer models. It also enables:

"release_axmodel_buffer_after_init": true

This releases each host-side axmodel initialization buffer after the runtime has created the model instance, reducing host-memory pressure during service startup.

Performance

The following reference measurements were collected on an AX637 / NPU1 development board with the packaged C128-P1152-CTX1535 runtime. TTFT is end-to-end time to first token, including media preprocessing and vision encoding where applicable.

Scenario	Input tokens	Prefill chunks	TTFT	Decode
Text-only smoke prompt	`25`	`1 x 128`	`1021.89 ms`	`n/a (short EOS reply)`
Image prompt	`97`	`1 x 128`	`1652.16 ms`	`n/a (short EOS reply)`
Raw video `.mp4`	`1140`	`9 x 128`	`22963.72 ms`	`6.20 token/s avg`
Video frames directory	`1140`	`9 x 128`	`18772.22 ms`	`6.95 token/s avg`

The video measurements use the packaged red-panda sample at an effective 2 FPS request rate. The runtime selects visual frames to remain within the shipped 1152-token multimodal tail budget.

Startup Runtime Footprint

Item	Value
Flash total (text + post + vision axmodels)	`1.70 GiB`
Package flash total	`2.40 GiB`

The tested board has 2 GiB CMM. Start the package from persistent board storage or NFS; do not copy the approximately 538 MiB vision axmodel to /tmp, which is commonly a tmpfs on AX637 images.

Vision Encoder Latency

Measured on AX637 / NPU1:

export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
/opt/bin/ax_run_model -m minicpmv4_6_vision_448.axmodel -w 5 -r 10

Model	Resolution	Soft Tokens	Time
`minicpmv4_6_vision_448.axmodel`	`448x448`	`64`	`544.119 ms avg`

The shipped visual encoder has a fixed profile:

vision_width = 448
vision_height = 448
vision_patch_size = 14
patch grid = 32 x 32
raw patch tokens = 1024
visual compression = 16x
soft tokens = 1024 / 16 = 64

Soft Tokens = (vision_width / patch_size) x (vision_height / patch_size) / 16

The visual soft-token count is fixed by the packaged encoder. Full request tokens can be larger because axllm includes the chat template and user text.

Package Layout

.
├── README.md
├── bin/
│   ├── axllm
│   └── axllm.version.json
├── lib/
├── assets/
│   ├── sample.png
│   ├── red-panda-openai.mp4
│   └── red-panda-openai-frames/
├── minicpmv4_6_vision_448.axmodel
├── qwen3_5_text_p128_l0_together.axmodel
├── ...
├── qwen3_5_text_p128_l23_together.axmodel
├── qwen3_5_text_post.axmodel
├── model.embed_tokens.weight.bfloat16.bin
├── config.json
├── post_config.json
└── minicpm_v46_tokenizer.txt

The runtime files are kept at the repository root so the package can be served directly with axllm serve ..

Sample Assets

Image: assets/sample.png
Raw video: assets/red-panda-openai.mp4
Extracted video frames: assets/red-panda-openai-frames/

Direct Inference with `axllm`

Download the Package

mkdir -p AXERA-TECH/MiniCPM-V-4.6-AX637
cd AXERA-TECH/MiniCPM-V-4.6-AX637
hf download AXERA-TECH/MiniCPM-V-4.6-AX637 --local-dir .

Install `axllm`

The validated runtime binary is included in this package:

chmod +x ./bin/axllm

Alternatively, install axllm from the public repository:

git clone -b axllm https://github.com/AXERA-TECH/ax-llm.git
cd ax-llm
./install.sh

Run on the Board

From the package root on AX637:

export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
./bin/axllm run .

In interactive mode, enter a text prompt directly. For video input, use video:/path/to/video.mp4 or video:/path/to/frames_dir.

Serve with `axllm`

export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
./bin/axllm serve . --port 8000

Expected model id:

AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16

Check readiness:

curl http://127.0.0.1:8000/health

Expected response:

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

The service exposes:

GET  /health
GET  /v1/models
POST /v1/chat/completions

Text Request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [{
      "role": "user",
      "content": [{"type": "text", "text": "What is 1+1? Reply with the number only."}]
    }],
    "max_tokens": 32
  }'

Example response:

{"choices":[{"message":{"role":"assistant","content":"2"},"finish_reason":"stop"}]}

Image Request

python3 - <<'PY'
import base64
import json
from pathlib import Path
from urllib.request import Request, urlopen

image = Path("assets/sample.png").read_bytes()
payload = {
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(image).decode()}},
        {"type": "text", "text": "What animal is the main subject in this image? Reply in Chinese only."}
    ]}],
    "max_tokens": 32,
}
request = Request(
    "http://127.0.0.1:8000/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
)
with urlopen(request, timeout=60) as response:
    print(response.read().decode())
PY

Validated response: 龙虾.

Video Request

axllm serve accepts a raw video file or an extracted frames directory through an image_url value prefixed with video:.

VIDEO_PATH="$(pwd)/assets/red-panda-openai.mp4"
curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "video:'"${VIDEO_PATH}"':2"}},
        {"type": "text", "text": "Describe this video briefly."}
      ]
    }],
    "max_tokens": 128
  }'

Validated raw-video response:

Two red pandas are playing with a rope. One is on a branch, the other is on the ground.

Raw video decoding requires ffprobe and ffmpeg in PATH. When they are unavailable, use the packaged frames directory instead:

video:/absolute/path/to/red-panda-openai-frames

The frames-directory request was also validated and returns a semantically equivalent description.

Browser UI with `lite_webui`

Use AXERA-TECH/lite_webui with the OpenAI base URL http://<board-ip>:8000 and model name AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

Original Hugging Face model: openbmb/MiniCPM-V-4.6
AXERA conversion and deployment workflow: AXERA-TECH/MiniCPM-V-4.6.axera

Discussion

GitHub Issues: https://github.com/AXERA-TECH/ax-llm/issues
QQ group: 139953715

Downloads last month: 10

Inference Providers NEW

Image-Text-to-Text

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AXERA-TECH/MiniCPM-V-4.6-AX637

Base model

openbmb/MiniCPM-V-4.6

Finetuned

(17)

this model