Kimi VL A3B

• Cheapest

  NVIDIA Tesla P40

  37tok/sest.mlx-8bit

  369 €

• Best value

  NVIDIA GeForce RTX 5070 Ti

  120tok/sest.mlx-6bit

  1.149 €

• Fastest

  NVIDIA GeForce RTX 5090

  190tok/sest.mlx-8bit

  6.028 €

• Mac

  Mac mini M4 Pro 48GB

  30tok/sest.mlx-8bit

  2.699 €

1 other card or machine we price cannot hold it at this context.

• mlx-5bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
• mlx-6bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
• mlx-8bit
  • LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-8bit
  • MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-8bit

MoE vision-language model with 16B total and about 3B active parameters; it needs trust_remote_code and was developed against transformers 4.48.2. vLLM support needed a branch from an unmerged pull request (#16387) when the card was written.

How much VRAM does Kimi VL A3B need?

At mlx-5bit, Kimi VL A3B needs about 12 GB of memory at 8k context and 12 GB at 8k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running Kimi VL A3B locally?

The cheapest hardware in Germany that runs Kimi VL A3B fully in memory at 8k context is the NVIDIA Tesla P40, 369 € at eBay, at about 37 tok/s (estimated). The best value is the NVIDIA GeForce RTX 5070 Ti (about 120 tok/s for 1.149 €). The fastest we price is the NVIDIA GeForce RTX 5090, at about 190 tok/s.

Can I run Kimi VL A3B on a 16 GB GPU?

Yes: at mlx-6bit it fits fully in 16 GB at 8k context. On the NVIDIA GeForce RTX 5080 it runs at about 128 tok/s (estimated).

Can I run Kimi VL A3B on a 24 GB GPU?

Yes: at mlx-8bit it fits fully in 24 GB at 8k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 107 tok/s (estimated).

What licence is Kimi VL A3B released under?

MIT, which allows commercial use.