AREX 2

• Cheapest

  NVIDIA GeForce RTX 3090

  27tok/sest.Q4_K_M

  1.500 €

• Best value

  NVIDIA GeForce RTX 3090

  27tok/sest.Q4_K_M

  1.500 €

• Fastest

  NVIDIA GeForce RTX 5090

  40tok/sest.Q6_K

  7.100 €

• Mac

  Mac Studio M3 Ultra 96GB

  16tok/sest.Q8_0

  5.999 €

1 other card or machine we price cannot hold it at this context.

• Q4_K_M
  • Ollamaollama run hf.co/bartowski/BAAI_AREX-2-GGUF:Q4_K_M
  • LM Studiolms get bartowski/BAAI_AREX-2-GGUF@q4_k_m
  • llama.cppllama-server -hf bartowski/BAAI_AREX-2-GGUF:Q4_K_M
• Q5_K_M
  • Ollamaollama run hf.co/bartowski/BAAI_AREX-2-GGUF:Q5_K_M
  • LM Studiolms get bartowski/BAAI_AREX-2-GGUF@q5_k_m
  • llama.cppllama-server -hf bartowski/BAAI_AREX-2-GGUF:Q5_K_M
• Q6_K
  • Ollamaollama run hf.co/bartowski/BAAI_AREX-2-GGUF:Q6_K
  • LM Studiolms get bartowski/BAAI_AREX-2-GGUF@q6_k
  • llama.cppllama-server -hf bartowski/BAAI_AREX-2-GGUF:Q6_K
• Q8_0
  • Ollamaollama run hf.co/bartowski/BAAI_AREX-2-GGUF:Q8_0
  • LM Studiolms get bartowski/BAAI_AREX-2-GGUF@q8_0
  • llama.cppllama-server -hf bartowski/BAAI_AREX-2-GGUF:Q8_0

Dense 27B multimodal model fine-tuned from Qwen3.8-27B; requires a recent Transformers release with Qwen3.8 support. The card says to follow the Qwen base model's terms and notices.

How much VRAM does AREX 2 need?

At Q4_K_M, AREX 2 needs about 17 GB of memory at 8k context and 21 GB at 64k, counting the weights, the KV cache and runtime buffers.

What is the best GPU for running AREX 2 locally?

The cheapest hardware in Germany that runs AREX 2 fully in memory at 64k context is the NVIDIA GeForce RTX 3090, 1.500 € at Amazon, at about 27 tok/s (estimated). The fastest we price is the NVIDIA GeForce RTX 5090, at about 40 tok/s.

What is the cheapest way to run AREX 2 in Germany today?

The NVIDIA GeForce RTX 3090, 1.500 € new at Amazon, in stock: it runs AREX 2 at Q4_K_M fully in memory at 64k context, at about 27 tok/s (estimated).

Can I run AREX 2 on a 16 GB GPU?

Only partly: at 64k context it does not fit in 16 GB, so part of it runs from system RAM and it slows to about 5 tok/s on the AMD Radeon RX 9060 XT 16GB (estimated).

Can I run AREX 2 on a 24 GB GPU?

Yes: at Q4_K_M it fits fully in 24 GB at 64k context. On the NVIDIA GeForce RTX 3090 Ti it runs at about 29 tok/s (estimated).

How do I run AREX 2 locally?

With Ollama: ollama run hf.co/bartowski/BAAI_AREX-2-GGUF:Q4_K_M (the Q4_K_M build). LM Studio and llama.cpp commands are on this page.

What licence is AREX 2 released under?

Apache License 2.0, which allows commercial use.