MiniCPM5 2B

• Cheapest

  NVIDIA GeForce RTX 3060 12GB

  55tok/sest.Q8_0

  309 €

• Best value

  NVIDIA GeForce RTX 3080 10GB

  115tok/sest.Q8_0

  500 €

• Fastest

  NVIDIA GeForce RTX 5090

  272tok/sest.Q8_0

  6.028 €

• Mac

  Mac mini M4 16GB

  19tok/sest.Q8_0

  800 €

• mlx-4bit
  • LM Studiolms get mlx-community/MiniCPM5-2B-mlx-4Bit
  • MLXmlx_lm.server --model mlx-community/MiniCPM5-2B-mlx-4Bit
• Q4_K_M
  • Ollamaollama run hf.co/bartowski/MiniCPM5-2B-GGUF:Q4_K_M
  • LM Studiolms get bartowski/MiniCPM5-2B-GGUF@q4_k_m
  • llama.cppllama-server -hf bartowski/MiniCPM5-2B-GGUF:Q4_K_M
• Q5_K_M
  • Ollamaollama run hf.co/bartowski/MiniCPM5-2B-GGUF:Q5_K_M
  • LM Studiolms get bartowski/MiniCPM5-2B-GGUF@q5_k_m
  • llama.cppllama-server -hf bartowski/MiniCPM5-2B-GGUF:Q5_K_M
• Q6_K
  • Ollamaollama run hf.co/bartowski/MiniCPM5-2B-GGUF:Q6_K
  • LM Studiolms get bartowski/MiniCPM5-2B-GGUF@q6_k
  • llama.cppllama-server -hf bartowski/MiniCPM5-2B-GGUF:Q6_K
• mlx-8bit
  • LM Studiolms get mlx-community/MiniCPM5-2B-8bit
  • MLXmlx_lm.server --model mlx-community/MiniCPM5-2B-8bit
• Q8_0
  • Ollamaollama run hf.co/bartowski/MiniCPM5-2B-GGUF:Q8_0
  • LM Studiolms get bartowski/MiniCPM5-2B-GGUF@q8_0
  • llama.cppllama-server -hf bartowski/MiniCPM5-2B-GGUF:Q8_0

Dense dual-language (en/zh) 2B model with standard LlamaForCausalLM architecture, aimed at on-device use. This is the final RL+OPD post-trained BF16 release; GGUF, MLX and GPTQ variants are separate repos.

How much VRAM does MiniCPM5 2B need?

At mlx-4bit, MiniCPM5 2B needs about 2 GB of memory at 8k context and 4 GB at 32k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running MiniCPM5 2B locally?

The cheapest hardware in Germany that runs MiniCPM5 2B fully in memory at 32k context is the NVIDIA GeForce RTX 3060 12GB, 309 € at Amazon, at about 55 tok/s (estimated). The best value is the NVIDIA GeForce RTX 3080 10GB (about 115 tok/s for 500 €). The fastest we price is the NVIDIA GeForce RTX 5090, at about 272 tok/s.

Can I run MiniCPM5 2B on a 16 GB GPU?

Yes: at Q8_0 it fits fully in 16 GB at 32k context. On the NVIDIA GeForce RTX 5080 it runs at about 146 tok/s (estimated).

Can I run MiniCPM5 2B on a 24 GB GPU?

Yes: at Q8_0 it fits fully in 24 GB at 32k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 153 tok/s (estimated).

How do I run MiniCPM5 2B locally?

With Ollama: ollama run hf.co/bartowski/MiniCPM5-2B-GGUF:Q4_K_M (the Q4_K_M build). LM Studio and llama.cpp commands are on this page.

What licence is MiniCPM5 2B released under?

Apache License 2.0, which allows commercial use.