DeepSeek R1 Distill Qwen 7B

• Cheapest

  NVIDIA GeForce RTX 5060 Ti 16GB

  20tok/sest.Q6_K

  £550

• Best value

  AMD Radeon RX 7800 XT

  25tok/sest.Q6_K

  £568

• Fastest

  NVIDIA GeForce RTX 5090

  71tok/sest.Q8_0

  £5,000

• Mac

  Mac Studio M4 Max 64GB

  23tok/sest.Q8_0

  £3,399

1 other card or machine we price cannot hold it at this context.

• mlx-3bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-7B-3bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-7B-3bit
• mlx-4bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-7B-4bit
• Q4_K_M
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF@q4_k_m
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M
• Q5_K_M
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q5_K_M
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF@q5_k_m
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q5_K_M
• mlx-6bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-7B-6bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-7B-6bit
• Q6_K
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q6_K
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF@q6_k
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q6_K
• mlx-8bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-7B-8bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-7B-8bit
• Q8_0
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q8_0
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF@q8_0
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q8_0

Distilled from DeepSeek-R1 onto Qwen2.5-Math-7B (Qwen2.5 series is Apache 2.0); the card licenses the weights under MIT. Use temperature 0.5-0.7, no system prompt, and start output with "<think>\n".

How much VRAM does DeepSeek R1 Distill Qwen 7B need?

At mlx-3bit, DeepSeek R1 Distill Qwen 7B needs about 4 GB of memory at 8k context and 12 GB at 128k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running DeepSeek R1 Distill Qwen 7B locally?

The cheapest hardware in the UK that runs DeepSeek R1 Distill Qwen 7B fully in memory at 128k context is the NVIDIA GeForce RTX 5060 Ti 16GB, £550 at eBay, at about 20 tok/s (estimated). The best value is the AMD Radeon RX 7800 XT (about 25 tok/s for £568). The fastest we price is the NVIDIA GeForce RTX 5090, at about 71 tok/s.

Can I run DeepSeek R1 Distill Qwen 7B on a 16 GB GPU?

Yes: at Q6_K it fits fully in 16 GB at 128k context. On the NVIDIA GeForce RTX 5080 it runs at about 43 tok/s (estimated).

Can I run DeepSeek R1 Distill Qwen 7B on a 24 GB GPU?

Yes: at Q8_0 it fits fully in 24 GB at 128k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 40 tok/s (estimated).

How do I run DeepSeek R1 Distill Qwen 7B locally?

With Ollama: ollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M (the Q4_K_M build). LM Studio and llama.cpp commands are on this page.

What licence is DeepSeek R1 Distill Qwen 7B released under?

MIT License, which allows commercial use.