What is the best GPU for running Qwen2.5 0.5B locally?

The cheapest hardware in the UK that runs Qwen2.5 0.5B fully in memory at 8k context is the NVIDIA Tesla P40, £261 at eBay, at about 340 tok/s (estimated). The best value is the NVIDIA GeForce RTX 3080 10GB (about 746 tok/s for £466). The fastest we price is the NVIDIA GeForce RTX 5090, at about 1759 tok/s.

Can I run Qwen2.5 0.5B on a 16 GB GPU?

Yes: at Q8_0 it fits fully in 16 GB at 8k context. On the NVIDIA GeForce RTX 5080 it runs at about 942 tok/s (estimated).

Can I run Qwen2.5 0.5B on a 24 GB GPU?

Yes: at Q8_0 it fits fully in 24 GB at 8k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 989 tok/s (estimated).

How do I run Qwen2.5 0.5B locally?

With Ollama: ollama run hf.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF:Q8_0 (the Q8_0 build). LM Studio and llama.cpp commands are on this page.

What licence is Qwen2.5 0.5B released under?

Apache License 2.0, which allows commercial use.

Qwen2.5 0.5B

494M parameters · 32k context · Apache License 2.0 · released 2024-09-16

chat

In short

Qwen2.5 0.5B is an open model from Alibaba Qwen with 494M parameters. At Q8_0, Qwen2.5 0.5B needs about 1 GB of memory at 8k context and 1 GB at 8k, counting the weights, the KV cache and runtime buffers. The cheapest hardware in the UK that runs Qwen2.5 0.5B fully in memory at 8k context is the NVIDIA Tesla P40, £261 at eBay, at about 340 tok/s (estimated).

Updated 15 October 2026. Speeds are estimates; how they are worked out.

What to run it on in the UK

Fully in memory at 8k context, priced at retailers today.

Cheapest
NVIDIA Tesla P40
340tok/sest.Q8_0
£261
Best value
NVIDIA GeForce RTX 3080 10GB
746tok/sest.Q8_0
£466
Fastest
NVIDIA GeForce RTX 5090
1759tok/sest.Q8_0
£5,000
Mac
Mac mini M4 16GB
124tok/sest.Q8_0
£765

All hardware that runs it

Everything we price, best value first. Two cards split the model by layers.

Hardware	Quant	Fits	Decode	Price	tok/s per 1,000
NVIDIA Tesla P40	Q8_0	fits	340 tok/s	£261	1302.6
AMD Radeon RX 7600 XT	Q8_0	fits	251 tok/s	£348	719.6
Intel Arc B580	Q8_0	fits	361 tok/s	£349	1035.5
NVIDIA GeForce RTX 3060 12GB	Q8_0	fits	353 tok/s	£390	905.9
NVIDIA GeForce RTX 3080 10GB	Q8_0	fits	746 tok/s	£466	1601.7
AMD Radeon RX 9060 XT 16GB	Q8_0	fits	279 tok/s	£499	558.6
NVIDIA GeForce RTX 5060 Ti 16GB	Q8_0	fits	440 tok/s	£550	799.5
AMD Radeon RX 7800 XT	Q8_0	fits	543 tok/s	£568	956.6
Intel Arc A770 16GB	Q8_0	fits	443 tok/s	£588	753.7
AMD Radeon RX 9070 XT	Q8_0	fits	557 tok/s	£670	832.0
NVIDIA GeForce RTX 5070	Q8_0	fits	660 tok/s	£695	948.9
Intel Arc Pro B60	Q8_0	fits	361 tok/s	£742	486.4
Mac mini M4 16GB	Q8_0	fits	124 tok/s	£765	161.4
AMD Radeon RX 7900 XT	Q8_0	fits	697 tok/s	£789	882.8
NVIDIA GeForce RTX 4070 Super	Q8_0	fits	495 tok/s	£874	566.1
NVIDIA GeForce RTX 4070 Ti Super	Q8_0	fits	660 tok/s	£950	694.2
NVIDIA GeForce RTX 5070 Ti	Q8_0	fits	879 tok/s	£975	901.9
NVIDIA GeForce RTX 4060 Ti 16GB	Q8_0	fits	283 tok/s	£977	289.4
AMD Radeon RX 7900 XTX	Q8_0	fits	836 tok/s	£999	836.6
NVIDIA GeForce RTX 5080	Q8_0	fits	942 tok/s	£1,199	785.8
Mac mini M4 32GB	Q8_0	fits	124 tok/s	£1,200	102.9
NVIDIA GeForce RTX 4080 Super	Q8_0	fits	722 tok/s	£1,429	505.3
AMD Radeon AI PRO R9700	Q8_0	fits	557 tok/s	£1,674	332.8
NVIDIA GeForce RTX 3090	Q8_0	fits	919 tok/s	£1,707	538.2
NVIDIA GeForce RTX 3090 Ti	Q8_0	fits	989 tok/s	£2,199	449.9
Mac mini M4 Pro 48GB	Q8_0	fits	281 tok/s	£3,000	93.6
GMKtec EVO-X2 Ryzen AI Max+ 395 128GB	Q8_0	fits	223 tok/s	£3,320	67.1
Mac Studio M4 Max 64GB	Q8_0	fits	562 tok/s	£3,399	165.3
NVIDIA GeForce RTX 5090	Q8_0	fits	1759 tok/s	£5,000	351.7
Mac Studio M3 Ultra 96GB	Q8_0	fits	843 tok/s	£5,499	153.2
NVIDIA GeForce RTX 4090	Q8_0	fits	989 tok/s	£6,015	164.5
Mac Studio M3 Ultra 256GB	Q8_0	fits	843 tok/s	£8,499	99.2
Mac Studio M3 Ultra 512GB	Q8_0	fits	843 tok/s	£17,495	48.2
NVIDIA RTX PRO 6000 Blackwell	Q8_0	fits	1759 tok/s	£29,504	59.6
Mac Studio M4 Max 128GB	Q8_0	fits	562 tok/s	–	–
Mac mini M4 Pro 64GB	Q8_0	fits	281 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 128GB	Q8_0	fits	223 tok/s	–	–
NVIDIA DGX Spark 128GB	Q8_0	fits	268 tok/s	–	–
ASUS Ascent GX10 128GB	Q8_0	fits	268 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 64GB	Q8_0	fits	223 tok/s	–	–

Memory needed

Weights, KV cache and runtime buffers at each context length (KV cache at fp16).

Quant	File	@8k	@32k
Q8_0gguf	0.5 GB	1.3 GB	1.8 GB

Run it

Copy a command for the runtime you use.

Q8_0
- Ollamaollama run hf.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF:Q8_0
- LM Studiolms get bartowski/Qwen2.5-0.5B-Instruct-GGUF@q8_0
- llama.cppllama-server -hf bartowski/Qwen2.5-0.5B-Instruct-GGUF:Q8_0

Notes

The card says the full context is 32,768 tokens and generation is up to 8192 tokens. Needs transformers 4.37.0 or later, because older versions raise KeyError: 'qwen2'.

Questions

How much VRAM does Qwen2.5 0.5B need?: At Q8_0, Qwen2.5 0.5B needs about 1 GB of memory at 8k context and 1 GB at 8k, counting the weights, the KV cache and runtime buffers.
What is the best GPU for running Qwen2.5 0.5B locally?: The cheapest hardware in the UK that runs Qwen2.5 0.5B fully in memory at 8k context is the NVIDIA Tesla P40, £261 at eBay, at about 340 tok/s (estimated). The best value is the NVIDIA GeForce RTX 3080 10GB (about 746 tok/s for £466). The fastest we price is the NVIDIA GeForce RTX 5090, at about 1759 tok/s.
Can I run Qwen2.5 0.5B on a 16 GB GPU?: Yes: at Q8_0 it fits fully in 16 GB at 8k context. On the NVIDIA GeForce RTX 5080 it runs at about 942 tok/s (estimated).
Can I run Qwen2.5 0.5B on a 24 GB GPU?: Yes: at Q8_0 it fits fully in 24 GB at 8k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 989 tok/s (estimated).
How do I run Qwen2.5 0.5B locally?: With Ollama: ollama run hf.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF:Q8_0 (the Q8_0 build). LM Studio and llama.cpp commands are on this page.
What licence is Qwen2.5 0.5B released under?: Apache License 2.0, which allows commercial use.