All models › ibm-granite

granite-switch-4.1-8b

ibm-granite · on Arena since 14 Oct 2026

Download New · waiting for Arena votes
Arena score—Not voted on Arena yet
Size8.8 Bparameters
4-bit download5.3 GBweights only, Q4_K_M
Max context128Ktokens, from the model config

Where to use it for free

New: waiting for Arena votes

Released on 14 Oct 2026. Arena has not voted on this model yet. As soon as it does, the model moves into the ranking automatically, usually the next night. Until then you can still check its size, memory needs and speed below.

Speed on popular hardware

HardwareGPU memoryFits?Speed
RTX 3060 12 GB12 GBFits in GPU23 tok/s
RTX 4060 Ti 16 GB16 GBFits in GPU18 tok/s
RTX 309024 GBFits in GPU60 tok/s
RTX 409024 GBFits in GPU64 tok/s
RTX 509032 GBFits in GPU114 tok/s
RX 7900 XTX24 GBFits in GPU61 tok/s
2 × RTX 309048 GBFits in GPU58 tok/s
4 × RTX 309096 GBFits in GPU55 tok/s
Mac M4 Max 128 GB96 GBFits in GPU35 tok/s
Ryzen AI Max+ 395 128 GB96 GBFits in GPU16 tok/s
8 × H100 SXM640 GBFits in GPU177 tok/s

Estimate at 4-bit with 32K context, 64 GB of DDR5 for offload. Speed ≈ memory bandwidth × 70% ÷ data read per token; real numbers vary by about ±30%.

GPU memory needed

Level8K context32K context128K contextQuality vs full
Q3_K_M6.4 GB11 GB28 GB+10.2% noticeable loss
Q4_K_M7.3 GB12 GB28 GB+2.7% small loss
Q5_K_M8.3 GB12 GB29 GB+0.9% very small loss
Q6_K9.2 GB13 GB30 GB+0.3% near-lossless
Q8_011 GB16 GB32 GB≈ 0% same as full
F1620 GB24 GB41 GB≈ 0% same as full

Weights + context cache (from the model's architecture) + runtime. Quality: increase in perplexity against the full model, from llama.cpp tests, scaled by model size. Simulate your own setup.