granite switch 4.1 8b preview
9.6B parameters · 128k context · Apache 2.0 · released 2026-05-05
chat
What to run it on in the US
Fully in memory at 128k context, priced at retailers today.
Nothing we price holds this model fully in memory.
All hardware that runs it
Everything we price, best value first. Two cards split the model by layers.
Nothing we price can run it at this context.
Memory needed
Weights, KV cache and runtime buffers at each context length (KV cache at fp16).
| Quant | File |
|---|
Run it
Copy a command for the runtime you use.
No published builds to run yet.
Notes
Preview model with 12 embedded LoRA adapters (RAG, safety, uncertainty) activated through chat-template control tokens; needs the granite-switch package to register the architecture. Requires Transformers >=5.5.1, and vLLM >=0.19.1, <0.21.0 if you use vLLM.
Questions
- What licence is granite switch 4.1 8b preview released under?
- Apache 2.0, which allows commercial use.