Can I run Kimi VL A3B on a 16 GB GPU?

Only partly: at 128k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 14 tok/s on the AMD Radeon RX 9060 XT 16GB (estimated).

Can I run Kimi VL A3B on a 24 GB GPU?

Yes: at mlx-8bit it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 51 tok/s (estimated).

What licence is Kimi VL A3B released under?

MIT, which allows commercial use.

Kimi VL A3B

16.4B total, 3B active · 128k context · MIT · released 2025-04-09

visionagenticchatreasoning

In short

Kimi VL A3B is an open model from Moonshot AI with 16.4B parameters (3B active per token). At mlx-5bit, Kimi VL A3B needs about 12 GB of memory at 8k context and 16 GB at 128k, counting the weights, the KV cache and runtime buffers. The cheapest hardware in Germany that runs Kimi VL A3B fully in memory at 128k context is the NVIDIA Tesla P40, 369 € at eBay, at about 18 tok/s (estimated).

Updated 15 October 2026. Speeds are estimates; how they are worked out.

What to run it on in Germany

Fully in memory at 128k context, priced at retailers today.

Cheapest
NVIDIA Tesla P40
18tok/sest.mlx-8bit
369 €
Best value
NVIDIA Tesla P40
18tok/sest.mlx-8bit
369 €
Fastest
NVIDIA GeForce RTX 5090
91tok/sest.mlx-8bit
6.028 €
Mac
Mac Studio M4 Max 64GB
29tok/sest.mlx-8bit
4.183 €

All hardware that runs it

Everything we price, best value first. Two cards split the model by layers.

Hardware	Quant	Fits	Decode	Price	tok/s per 1,000
NVIDIA Tesla P40	mlx-8bit	fits	18 tok/s	369 €	47.4
AMD Radeon RX 7900 XT	mlx-6bit	fits	40 tok/s	855 €	46.8
Intel Arc Pro B60	mlx-8bit	fits	17 tok/s	1.074 €	15.9
Mac mini M4 32GB	mlx-6bit	fits	7 tok/s	1.400 €	5.1
AMD Radeon RX 7900 XTX	mlx-8bit	fits	43 tok/s	1.555 €	27.8
NVIDIA GeForce RTX 3090	mlx-8bit	fits	48 tok/s	1.850 €	25.7
AMD Radeon AI PRO R9700	mlx-8bit	fits	29 tok/s	1.867 €	15.4
NVIDIA GeForce RTX 3090 Ti	mlx-8bit	fits	51 tok/s	2.500 €	20.4
Mac mini M4 Pro 48GB	mlx-8bit	fits	15 tok/s	2.699 €	5.4
Mac mini M4 Pro 64GB	mlx-8bit	fits	15 tok/s	2.799 €	5.2
GMKtec EVO-X2 Ryzen AI Max+ 395 128GB	mlx-8bit	fits	12 tok/s	3.700 €	3.1
NVIDIA GeForce RTX 4090	mlx-8bit	fits	51 tok/s	3.955 €	12.9
Mac Studio M4 Max 64GB	mlx-8bit	fits	29 tok/s	4.183 €	6.9
Mac Studio M4 Max 128GB	mlx-8bit	fits	29 tok/s	5.800 €	5.0
Mac Studio M3 Ultra 96GB	mlx-8bit	fits	44 tok/s	5.999 €	7.3
NVIDIA GeForce RTX 5090	mlx-8bit	fits	91 tok/s	6.028 €	15.1
NVIDIA DGX Spark 128GB	mlx-8bit	fits	14 tok/s	8.313 €	1.7
ASUS Ascent GX10 128GB	mlx-8bit	fits	14 tok/s	8.623 €	1.6
Mac Studio M3 Ultra 256GB	mlx-8bit	fits	44 tok/s	15.555 €	2.8
Mac Studio M3 Ultra 512GB	mlx-8bit	fits	44 tok/s	18.995 €	2.3
NVIDIA RTX PRO 6000 Blackwell	mlx-8bit	fits	91 tok/s	19.076 €	4.8
Framework Desktop Ryzen AI Max+ 395 128GB	mlx-8bit	fits	12 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 64GB	mlx-8bit	fits	12 tok/s	–	–
NVIDIA GeForce RTX 3060 12GB	mlx-5bit	offload	11 tok/s	309 €	34.0
Intel Arc B580	mlx-5bit	offload	9 tok/s	360 €	26.1
NVIDIA GeForce RTX 3080 10GB	mlx-5bit	offload	10 tok/s	500 €	19.8
AMD Radeon RX 9060 XT 16GB	mlx-5bit	offload	14 tok/s	550 €	26.2
Intel Arc A770 16GB	mlx-5bit	offload	19 tok/s	646 €	28.7
NVIDIA GeForce RTX 4060 Ti 16GB	mlx-5bit	offload	15 tok/s	650 €	23.2
NVIDIA GeForce RTX 5070	mlx-5bit	offload	13 tok/s	699 €	17.9
NVIDIA GeForce RTX 4070 Super	mlx-5bit	offload	12 tok/s	707 €	16.4
NVIDIA GeForce RTX 5060 Ti 16GB	mlx-5bit	offload	22 tok/s	739 €	29.1
AMD Radeon RX 9070 XT	mlx-5bit	offload	24 tok/s	778 €	30.3
AMD Radeon RX 7600 XT	mlx-5bit	offload	13 tok/s	813 €	16.2
AMD Radeon RX 7800 XT	mlx-5bit	offload	23 tok/s	1.002 €	23.2
NVIDIA GeForce RTX 4080 Super	mlx-5bit	offload	31 tok/s	1.100 €	27.7
NVIDIA GeForce RTX 5070 Ti	mlx-5bit	offload	35 tok/s	1.149 €	30.1
NVIDIA GeForce RTX 4070 Ti Super	mlx-5bit	offload	29 tok/s	1.247 €	23.0
NVIDIA GeForce RTX 5080	mlx-5bit	offload	36 tok/s	1.500 €	24.0

1 other card or machine we price cannot hold it at this context.

Memory needed

Weights, KV cache and runtime buffers at each context length (KV cache at fp16).

Quant	File	@8k	@32k	@64k	@128k
mlx-5bitmlx	11.0 GB	12.0 GB	12.9 GB	14.0 GB	16.3 GB
mlx-6bitmlx	12.9 GB	13.9 GB	14.7 GB	15.9 GB	18.1 GB
mlx-8bitmlx	16.6 GB	17.6 GB	18.4 GB	19.6 GB	21.8 GB

Run it

Copy a command for the runtime you use.

mlx-5bit
- LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
- MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-5bit
mlx-6bit
- LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
- MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-6bit
mlx-8bit
- LM Studiolms get mlx-community/Kimi-VL-A3B-Thinking-2506-8bit
- MLXmlx_lm.server --model mlx-community/Kimi-VL-A3B-Thinking-2506-8bit

Notes

MoE vision-language model with 16B total and about 3B active parameters; it needs trust_remote_code and was developed against transformers 4.48.2. vLLM support needed a branch from an unmerged pull request (#16387) when the card was written.

Questions

How much VRAM does Kimi VL A3B need?: At mlx-5bit, Kimi VL A3B needs about 12 GB of memory at 8k context and 16 GB at 128k, counting the weights, the KV cache and runtime buffers.
What is the best GPU for running Kimi VL A3B locally?: The cheapest hardware in Germany that runs Kimi VL A3B fully in memory at 128k context is the NVIDIA Tesla P40, 369 € at eBay, at about 18 tok/s (estimated). The fastest we price is the NVIDIA GeForce RTX 5090, at about 91 tok/s.
Can I run Kimi VL A3B on a 16 GB GPU?: Only partly: at 128k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 14 tok/s on the AMD Radeon RX 9060 XT 16GB (estimated).
Can I run Kimi VL A3B on a 24 GB GPU?: Yes: at mlx-8bit it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 51 tok/s (estimated).
What licence is Kimi VL A3B released under?: MIT, which allows commercial use.