What is the best GPU for running GLM 4.1V 9B Thinking locally?

The cheapest hardware in Canada that runs GLM 4.1V 9B Thinking fully in memory at 128k context is the AMD Radeon RX 7900 XT, $2,083 at Amazon, at about 26 tok/s (estimated). The best value is the NVIDIA RTX PRO 6000 Blackwell (about 65 tok/s for $3,133). The fastest we price is the NVIDIA RTX PRO 6000 Blackwell, at about 65 tok/s.

Can I run GLM 4.1V 9B Thinking on a 16 GB GPU?

Only partly: at 128k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 10 tok/s on the Intel Arc A770 16GB (estimated).

Can I run GLM 4.1V 9B Thinking on a 24 GB GPU?

Yes: at mlx-8bit it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 37 tok/s (estimated).

What licence is GLM 4.1V 9B Thinking released under?

MIT, which allows commercial use.

GLM 4.1V 9B Thinking

10.3B parameters · 64k context · MIT · released 2025-06-28

visionreasoningchat

In short

The cheapest hardware in Canada that runs GLM 4.1V 9B Thinking fully in memory at 128k context is the AMD Radeon RX 7900 XT, $2,083 at Amazon, at about 26 tok/s (estimated).

Updated October 15, 2026. Speeds are estimates; how they are worked out.

What to run it on in Canada

Fully in memory at 128k context, priced at retailers today.

Cheapest
AMD Radeon RX 7900 XT
26tok/sest.mlx-8bit
$2,083
Best value
NVIDIA RTX PRO 6000 Blackwell
65tok/sest.mlx-8bit
$3,133
Fastest
NVIDIA RTX PRO 6000 Blackwell
65tok/sest.mlx-8bit
$3,133

All hardware that runs it

Everything we price, best value first. Two cards split the model by layers.

Hardware	Quant	Fits	Decode	Price	tok/s per 1,000
Intel Arc Pro B60	mlx-8bit	fits	13 tok/s	$1,060	12.5
AMD Radeon RX 7900 XT	mlx-8bit	fits	26 tok/s	$2,083	12.3
NVIDIA GeForce RTX 3090	mlx-8bit	fits	34 tok/s	$2,800	12.1
AMD Radeon AI PRO R9700	mlx-8bit	fits	21 tok/s	$2,800	7.3
NVIDIA RTX PRO 6000 Blackwell	mlx-8bit	fits	65 tok/s	$3,133	20.7
NVIDIA GeForce RTX 3090 Ti	mlx-8bit	fits	37 tok/s	$3,999	9.1
GMKtec EVO-X2 Ryzen AI Max+ 395 128GB	mlx-8bit	fits	8 tok/s	$5,099	1.6
NVIDIA GeForce RTX 4090	mlx-8bit	fits	37 tok/s	$7,107	5.1
ASUS Ascent GX10 128GB	mlx-8bit	fits	10 tok/s	$8,299	1.2
NVIDIA GeForce RTX 5090	mlx-8bit	fits	65 tok/s	$10,300	6.3
Mac Studio M4 Max 128GB	mlx-8bit	fits	21 tok/s	–	–
Mac mini M4 Pro 64GB	mlx-8bit	fits	10 tok/s	–	–
AMD Radeon RX 7900 XTX	mlx-8bit	fits	31 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 128GB	mlx-8bit	fits	8 tok/s	–	–
Mac Studio M3 Ultra 512GB	mlx-8bit	fits	31 tok/s	–	–
NVIDIA DGX Spark 128GB	mlx-8bit	fits	10 tok/s	–	–
Mac Studio M3 Ultra 256GB	mlx-8bit	fits	31 tok/s	–	–
Mac mini M4 Pro 48GB	mlx-8bit	fits	10 tok/s	–	–
Mac Studio M4 Max 64GB	mlx-8bit	fits	21 tok/s	–	–
Mac Studio M3 Ultra 96GB	mlx-8bit	fits	31 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 64GB	mlx-8bit	fits	8 tok/s	–	–
Mac mini M4 32GB	mlx-8bit	fits	5 tok/s	–	–
NVIDIA Tesla P40	mlx-8bit	fits	13 tok/s	–	–
NVIDIA GeForce RTX 3060 12GB	mlx-8bit	offload	6 tok/s	$497	12.1
Intel Arc B580	mlx-8bit	offload	6 tok/s	$560	10.4
Intel Arc A770 16GB	mlx-8bit	offload	10 tok/s	$655	15.4
NVIDIA GeForce RTX 3080 10GB	mlx-8bit	offload	6 tok/s	$769	7.3
AMD Radeon RX 9060 XT 16GB	mlx-8bit	offload	8 tok/s	$800	9.6
AMD Radeon RX 9070 XT	mlx-8bit	offload	11 tok/s	$1,050	10.9
NVIDIA GeForce RTX 5060 Ti 16GB	mlx-8bit	offload	11 tok/s	$1,120	9.5
NVIDIA GeForce RTX 5070	mlx-8bit	offload	7 tok/s	$1,181	5.8
AMD Radeon RX 7800 XT	mlx-8bit	offload	11 tok/s	$1,397	8.0
NVIDIA GeForce RTX 4070 Super	mlx-8bit	offload	7 tok/s	$1,400	4.6
AMD Radeon RX 7600 XT	mlx-8bit	offload	7 tok/s	$1,440	5.0
NVIDIA GeForce RTX 5070 Ti	mlx-8bit	offload	15 tok/s	$1,690	8.8
NVIDIA GeForce RTX 4070 Ti Super	mlx-8bit	offload	13 tok/s	$1,790	7.3
NVIDIA GeForce RTX 4080 Super	mlx-8bit	offload	14 tok/s	$2,000	6.9
NVIDIA GeForce RTX 5080	mlx-8bit	offload	15 tok/s	$2,370	6.5
NVIDIA GeForce RTX 4060 Ti 16GB	mlx-8bit	offload	8 tok/s	–	–

1 other card or machine we price cannot hold it at this context.

Memory needed

Weights, KV cache and runtime buffers at each context length (KV cache at fp16).

Quant	File	@8k	@32k	@64k
mlx-8bitmlx	11.0 GB	12.0 GB	13.1 GB	14.5 GB

Run it

Copy a command for the runtime you use.

mlx-8bit
- LM Studiolms get mlx-community/GLM-4.1V-9B-Thinking-8bit
- MLXmlx_lm.server --model mlx-community/GLM-4.1V-9B-Thinking-8bit

Notes

Needs transformers 4.57.1 or later. It is a 9B bilingual (Chinese/English) vision-language reasoning model that handles images up to 4K resolution at any aspect ratio.

Questions

What is the best GPU for running GLM 4.1V 9B Thinking locally?: The cheapest hardware in Canada that runs GLM 4.1V 9B Thinking fully in memory at 128k context is the AMD Radeon RX 7900 XT, $2,083 at Amazon, at about 26 tok/s (estimated). The best value is the NVIDIA RTX PRO 6000 Blackwell (about 65 tok/s for $3,133). The fastest we price is the NVIDIA RTX PRO 6000 Blackwell, at about 65 tok/s.
Can I run GLM 4.1V 9B Thinking on a 16 GB GPU?: Only partly: at 128k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 10 tok/s on the Intel Arc A770 16GB (estimated).
Can I run GLM 4.1V 9B Thinking on a 24 GB GPU?: Yes: at mlx-8bit it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 37 tok/s (estimated).
What licence is GLM 4.1V 9B Thinking released under?: MIT, which allows commercial use.