Kimi VL A3B

• Cheapest

  NVIDIA Tesla P40

  30tok/sest.mlx-8bit

  369 €

• Best value

  NVIDIA GeForce RTX 5070 Ti

  94tok/sest.mlx-6bit

  1.149 €

• Fastest

  NVIDIA GeForce RTX 5090

  156tok/sest.mlx-8bit

  6.028 €

• Mac

  Mac mini M4 Pro 48GB

  25tok/sest.mlx-8bit

  2.699 €

1 other card or machine we price cannot hold it at this context.

• mlx-5bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
• mlx-6bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
• mlx-8bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-8bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-8bit

MoE vision-language model with 16B total and about 3B active parameters; it needs trust_remote_code and was developed against transformers 4.48.2. vLLM support needed a branch from an unmerged pull request (#16387) when the card was written.

How much VRAM does Kimi VL A3B need?

At mlx-5bit, Kimi VL A3B needs about 12 GB of memory at 8k context and 13 GB at 32k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running Kimi VL A3B locally?

The cheapest hardware in Germany that runs Kimi VL A3B fully in memory at 32k context is the NVIDIA Tesla P40, 369 € at eBay, at about 30 tok/s (estimated). The best value is the NVIDIA GeForce RTX 5070 Ti (about 94 tok/s for 1.149 €). The fastest we price is the NVIDIA GeForce RTX 5090, at about 156 tok/s.

Can I run Kimi VL A3B on a 16 GB GPU?

Yes: at mlx-6bit it fits fully in 16 GB at 32k context. On the NVIDIA GeForce RTX 5080 it runs at about 101 tok/s (estimated).

Can I run Kimi VL A3B on a 24 GB GPU?

Yes: at mlx-8bit it fits fully in 24 GB at 32k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 88 tok/s (estimated).

What licence is Kimi VL A3B released under?

MIT, which allows commercial use.