gpt-oss 20B

20.9B total, 4.2B active · 128k context · apache-2.0 · released 2025-08-04

chatcodingagenticreasoning

What to run it on in India

Fully in memory at 32k context, priced at retailers today.

Something we price holds it fully in memory, but none reaches 15 tok/s, our bar for recommending it. The table below has the speeds.

All hardware that runs it

Everything we price, best value first. Two cards split the model by layers.

Hardware	Quant	Fits	Decode	Price	tok/s per 1,000
NVIDIA GeForce RTX 3090	MXFP4	fits	108 tok/s	–	–
NVIDIA GeForce RTX 5090	MXFP4	fits	206 tok/s	–	–
Mac Studio M4 Max 128GB	MXFP4	fits	66 tok/s	–	–
NVIDIA GeForce RTX 4090	MXFP4	fits	116 tok/s	–	–
NVIDIA GeForce RTX 5070 Ti	MXFP4	fits	103 tok/s	–	–
Mac mini M4 Pro 64GB	MXFP4	fits	33 tok/s	–	–
NVIDIA GeForce RTX 5060 Ti 16GB	MXFP4	fits	52 tok/s	–	–
AMD Radeon RX 7900 XTX	MXFP4	fits	98 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 128GB	MXFP4	fits	26 tok/s	–	–
Mac Studio M3 Ultra 512GB	MXFP4	fits	99 tok/s	–	–
NVIDIA DGX Spark 128GB	MXFP4	fits	31 tok/s	–	–
AMD Radeon RX 9070 XT	MXFP4	fits	65 tok/s	–	–
GMKtec EVO-X2 Ryzen AI Max+ 395 128GB	MXFP4	fits	26 tok/s	–	–
Mac Studio M3 Ultra 256GB	MXFP4	fits	99 tok/s	–	–
Mac mini M4 Pro 48GB	MXFP4	fits	33 tok/s	–	–
NVIDIA GeForce RTX 4060 Ti 16GB	MXFP4	fits	33 tok/s	–	–
NVIDIA GeForce RTX 5080	MXFP4	fits	110 tok/s	–	–
Mac Studio M4 Max 64GB	MXFP4	fits	66 tok/s	–	–
NVIDIA GeForce RTX 4070 Ti Super	MXFP4	fits	77 tok/s	–	–
AMD Radeon RX 9060 XT 16GB	MXFP4	fits	33 tok/s	–	–
Mac Studio M3 Ultra 96GB	MXFP4	fits	99 tok/s	–	–
NVIDIA GeForce RTX 4080 Super	MXFP4	fits	85 tok/s	–	–
NVIDIA RTX PRO 6000 Blackwell	MXFP4	fits	206 tok/s	–	–
AMD Radeon AI PRO R9700	MXFP4	fits	65 tok/s	–	–
AMD Radeon RX 7800 XT	MXFP4	fits	64 tok/s	–	–
AMD Radeon RX 7900 XT	MXFP4	fits	82 tok/s	–	–
ASUS Ascent GX10 128GB	MXFP4	fits	31 tok/s	–	–
Framework Desktop Ryzen AI Max+ 395 64GB	MXFP4	fits	26 tok/s	–	–
Mac mini M4 32GB	MXFP4	fits	15 tok/s	–	–
NVIDIA GeForce RTX 3090 Ti	MXFP4	fits	116 tok/s	–	–
AMD Radeon RX 7600 XT	MXFP4	fits	29 tok/s	–	–
Intel Arc Pro B60	MXFP4	fits	39 tok/s	–	–
NVIDIA Tesla P40	MXFP4	fits	40 tok/s	–	–
Intel Arc A770 16GB	MXFP4	fits	48 tok/s	–	–
NVIDIA GeForce RTX 3060 12GB	MXFP4	offload	30 tok/s	–	–
NVIDIA GeForce RTX 5070	MXFP4	offload	42 tok/s	–	–
NVIDIA GeForce RTX 4070 Super	MXFP4	offload	36 tok/s	–	–
Intel Arc B580	MXFP4	offload	26 tok/s	–	–
NVIDIA GeForce RTX 3080 10GB	MXFP4	offload	27 tok/s	–	–

1 other card or machine we price cannot hold it at this context.

Memory needed

Weights, KV cache and runtime buffers at each context length (KV cache at fp16).

Quant	File	@8k	@32k	@64k	@128k
MXFP4gguf	11.3 GB	12.2 GB	12.9 GB	13.9 GB	15.7 GB

Run it

Copy a command for the runtime you use.

MXFP4
- Ollamaollama run hf.co/ggml-org/gpt-oss-20b-GGUF:MXFP4
- LM Studiolms get ggml-org/gpt-oss-20b-GGUF@mxfp4
- llama.cppllama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4