DeepSeek R1 Distill Qwen 14B

• Cheapest

  AMD Radeon RX 7900 XTX

  23tok/sest.Q5_K_M

  1.555 €

• Best value

  AMD Radeon RX 7900 XTX

  23tok/sest.Q5_K_M

  1.555 €

• Fastest

  NVIDIA GeForce RTX 5090

  39tok/sest.Q8_0

  6.028 €

• Mac

  Mac Studio M3 Ultra 96GB

  19tok/sest.Q8_0

  5.999 €

2 other cards and machines we price cannot hold it at this context.

• mlx-3bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-14B-3bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-3bit
• mlx-4bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit
• Q4_K_M
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF@q4_k_m
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M
• Q5_K_M
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q5_K_M
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF@q5_k_m
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q5_K_M
• mlx-6bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-14B-6bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-6bit
• Q6_K
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q6_K
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF@q6_k
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q6_K
• mlx-8bit
  • LM Studiolms get mlx-community/DeepSeek-R1-Distill-Qwen-14B-8bit
  • MLXmlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-8bit
• Q8_0
  • Ollamaollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q8_0
  • LM Studiolms get unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF@q8_0
  • llama.cppllama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q8_0

Distilled from DeepSeek-R1 onto Qwen2.5-14B (originally Apache 2.0); card says to avoid system prompts and use temperature 0.6. Card recommends forcing output to start with "<think>\n".

How much VRAM does DeepSeek R1 Distill Qwen 14B need?

At mlx-3bit, DeepSeek R1 Distill Qwen 14B needs about 8 GB of memory at 8k context and 19 GB at 64k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running DeepSeek R1 Distill Qwen 14B locally?

The cheapest hardware in Germany that runs DeepSeek R1 Distill Qwen 14B fully in memory at 64k context is the AMD Radeon RX 7900 XTX, 1.555 € at eBay, at about 23 tok/s (estimated). The fastest we price is the NVIDIA GeForce RTX 5090, at about 39 tok/s.

Can I run DeepSeek R1 Distill Qwen 14B on a 16 GB GPU?

Only partly: at 64k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 5 tok/s on the AMD Radeon RX 9060 XT 16GB (estimated).

Can I run DeepSeek R1 Distill Qwen 14B on a 24 GB GPU?

Yes: at Q5_K_M it fits fully in 24 GB at 64k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 27 tok/s (estimated).

How do I run DeepSeek R1 Distill Qwen 14B locally?

With Ollama: ollama run hf.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M (the Q4_K_M build). LM Studio and llama.cpp commands are on this page.

What licence is DeepSeek R1 Distill Qwen 14B released under?

MIT License, which allows commercial use.