XiaomiMiMo/MiMo-V2.6-Flash-RL

Xiaomi's efficiency-balanced omnimodal MoE (309B total / 15B active) with hybrid attention, FP8-compute/mxfp4-stored weights, 1M context, and a DFlash speculative decoder — the default MOPD checkpoint fixes the RL release's tool-call repetition

View on HuggingFace

moe309B / 15B1,048,576 ctxvLLM 0.31.0+textmultimodal

Guide

Overview

This recipe serves the MiMo-V2.6-Flash family from two checkpoints on the Variant row:

MOPD (default) — XiaomiMiMo/MiMo-V2.6-Flash-MOPD, the MOPD2-distilled upgrade and the recommended starting point for new deployments. It fixes the RL release's most visible agentic failure mode — tool-call repetition, where the model keeps re-issuing the same or highly similar tool calls while making no progress (the blog measures the Flash line as the more affected of the two) — with benchmark quality unchanged, folded into a regular MOPD2 pass via a short specialized-teacher run (diagnosis blog).
RL (original) — XiaomiMiMo/MiMo-V2.6-Flash-RL, the 2026-09-22 RL release, kept for reproducing earlier results. It can get stuck re-issuing the same tool calls in agentic workloads; prefer MOPD unless you are reproducing a specific earlier run.

MOPD2 (Technical Report §5.6) is multi-teacher on-policy distillation: domain-specialized mixRL and SFT teachers supervise the student on three on-policy streams (standard MOPD, teacher-prefix OPD, and SFT-prefix OPD).

The architecture is shared by both checkpoints: a sparse MoE model with 309B total parameters and 15B active per token — 48 layers (1 dense + 47 MoE) with 256 routed experts (top-8), hybrid attention (sliding-window 128 and global attention), and a 5-layer DFlash-style MTP drafter that predicts 7 tokens per pass. The model is natively omnimodal (text, image, video, audio via a 681M-param MiMo ViT and audio encoders) and supports up to 1M tokens of context.

Weights are stored as mxfp4 and computed as FP8 (block-wise e4m3, 128x128), so the checkpoint is only 173 GB on disk. Loading this mixed storage format requires vLLM with the MiMo V2 mxfp4/bf16-router support (vllm-project/vllm#57784), first shipped in the v0.31.0 stable release.

Prerequisites

Hardware: 4x H200 (TP4), 4x MI355X/MI325X/MI300X (TP4), or equivalent aggregate VRAM (>= 208 GB)

Pull the vLLM docker image

vLLM 0.31.0 is the first stable release that can load the mxfp4-stored weights (vllm-project/vllm#57784); earlier stable releases cannot:

docker pull vllm/vllm-openai:v0.31.0

MI300X / MI325X (gfx942) additionally need the MiMo MXFP4 routing gate (vllm-project/vllm#58262), which merged after the v0.31.0 branch cut — on those GPUs use a nightly wheel built after 2026-09-30 (uv pip install -U vllm --extra-index-url https://wheels.vllm.ai/nightly/cu130) or the ROCm nightly image.

This release also registers the mimo tool-call and reasoning parsers with the Rust frontend (vllm-project/vllm#57933).

The Hugging Face model cards point at vllm/vllm-openai:mimov25-cu129, the V2.5-era image; that tag predates the mxfp4-stored-weight support linked above, so use the v0.31.0 image recommended above.

Launch command

Single-node TP4 (H200). Selecting the RL (original) variant in the builder swaps only the checkpoint id (XiaomiMiMo/MiMo-V2.6-Flash-RL); everything else is identical:

vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

AMD (MI355X, gfx950)

VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \
vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

MI355X also runs TP2 just fine.

AMD (MI300X / MI325X, gfx942)

gfx942 has no native MXFP4 MoE path: without an explicit backend vLLM's oracle falls back to OCP emulation — correct, but about 6.5x lower output throughput. Select AITER's Triton W4A16 kernel instead with --moe-backend aiter_triton_mxfp4_bf16; supported since vllm-project/vllm#58262 (merged 2026-09-30, after the v0.31.0 branch cut — not in any stable release yet, so this flag needs a nightly wheel; it also needs aiter >= 0.1.20, which the ROCm nightly image ships). Verified end-to-end on MI300X in that PR — see it for the gsm8k and vllm bench serve numbers against the emulation baseline:

VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \
vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --moe-backend aiter_triton_mxfp4_bf16 \
  --generation-config vllm

With DFlash speculative decoding (drafter ships inside the checkpoint). vLLM does not resolve a <repo>/dflash Hub subfolder as the draft model — model must be a concrete local path to the dflash/ directory inside the downloaded snapshot. Resolve it per checkpoint (run hf download <checkpoint> first if it is not cached yet):

# MOPD variant
DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-MOPD/snapshots/*/dflash)
# RL (original) variant
# DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-RL/snapshots/*/dflash)
echo "$DFLASH_DIR"

vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --speculative-config "{\"method\":\"dflash\",\"model\":\"$DFLASH_DIR\",\"num_speculative_tokens\":7}" \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

The drafter inherits the target's tensor-parallel size by default. To pin it explicitly, add "draft_tensor_parallel_size": 4 to the speculative config (useful if you want the draft on fewer GPUs than the target, or to silence mis-detection on asymmetric topologies).

Client Usage

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "XiaomiMiMo/MiMo-V2.6-Flash-MOPD",
    "messages": [{"role": "user", "content": "Hello MiMo!"}],
    "temperature": 1.0,
    "top_p": 0.95,
    "chat_template_kwargs": {"enable_thinking": true}
  }'

Recommended sampling: temperature=1.0, top_p=0.95. Set "enable_thinking": false (or omit the kwargs) to disable thinking mode.