granite 4.2 3b

• Cheapest

  NVIDIA GeForce RTX 5060 Ti 16GB

  19tok/sest.Q8_0

  £550

• Best value

  AMD Radeon RX 7800 XT

  24tok/sest.Q8_0

  £568

• Fastest

  NVIDIA GeForce RTX 5090

  76tok/sest.Q8_0

  £5,000

• Mac

  Mac Studio M4 Max 64GB

  24tok/sest.Q8_0

  £3,399

1 other card or machine we price cannot hold it at this context.

• Q4_K_M
  • Ollamaollama run hf.co/bartowski/granite-4.2-3b-GGUF:Q4_K_M
  • LM Studiolms get bartowski/granite-4.2-3b-GGUF@q4_k_m
  • llama.cppllama-server -hf bartowski/granite-4.2-3b-GGUF:Q4_K_M
• Q5_K_M
  • Ollamaollama run hf.co/bartowski/granite-4.2-3b-GGUF:Q5_K_M
  • LM Studiolms get bartowski/granite-4.2-3b-GGUF@q5_k_m
  • llama.cppllama-server -hf bartowski/granite-4.2-3b-GGUF:Q5_K_M
• Q6_K
  • Ollamaollama run hf.co/bartowski/granite-4.2-3b-GGUF:Q6_K
  • LM Studiolms get bartowski/granite-4.2-3b-GGUF@q6_k
  • llama.cppllama-server -hf bartowski/granite-4.2-3b-GGUF:Q6_K
• Q8_0
  • Ollamaollama run hf.co/bartowski/granite-4.2-3b-GGUF:Q8_0
  • LM Studiolms get bartowski/granite-4.2-3b-GGUF@q8_0
  • llama.cppllama-server -hf bartowski/granite-4.2-3b-GGUF:Q8_0

Native context is 128K, and the card also claims a long-context extension to 512K; its serving examples use 131072. The vLLM reasoning parser needs vLLM v0.20+ (or SGLang v0.5.18+), and the recommended sampling is temperature=1.0 and top_p=0.95.

How much VRAM does granite 4.2 3b need?

At Q4_K_M, granite 4.2 3b needs about 4 GB of memory at 8k context and 14 GB at 128k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running granite 4.2 3b locally?

The cheapest hardware in the UK that runs granite 4.2 3b fully in memory at 128k context is the NVIDIA GeForce RTX 5060 Ti 16GB, £550 at eBay, at about 19 tok/s (estimated). The best value is the AMD Radeon RX 7800 XT (about 24 tok/s for £568). The fastest we price is the NVIDIA GeForce RTX 5090, at about 76 tok/s.

Can I run granite 4.2 3b on a 16 GB GPU?

Yes: at Q8_0 it fits fully in 16 GB at 128k context. On the NVIDIA GeForce RTX 5080 it runs at about 41 tok/s (estimated).

Can I run granite 4.2 3b on a 24 GB GPU?

Yes: at Q8_0 it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 43 tok/s (estimated).

How do I run granite 4.2 3b locally?

With Ollama: ollama run hf.co/bartowski/granite-4.2-3b-GGUF:Q4_K_M (the Q4_K_M build). LM Studio and llama.cpp commands are on this page.

What licence is granite 4.2 3b released under?

Apache 2.0, which allows commercial use.