Google
Google
Google/Gemma 4 31B IT

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q3_k_m	Unsloth Unsloth	13.7 GB	13.6 tok/s	153.4 tok/s	Runs poorly
q4_k_m	Unsloth Unsloth	17.1 GB	16.9 tok/s	159.2 tok/s	Runs poorly
4bit	MLX Community MLX Community	17.1 GB	N/A	N/A	N/A
q4_k_m	ggml ggml	17.4 GB	N/A	N/A	N/A
ud-q4_k_xl	Unsloth Unsloth	17.5 GB	N/A	N/A	N/A
q5_k_m	Unsloth Unsloth	20.2 GB	13.0 tok/s	147.6 tok/s	Runs poorly
4bit	Unsloth Unsloth	21.7 GB	14.3 tok/s	157.6 tok/s	Barely runs
6bit	MLX Community MLX Community	24.3 GB	N/A	N/A	N/A
4bit	LM Studio LM Studio	26.8 GB	N/A	N/A	N/A
q8_0	Unknown	30.4 GB	N/A	N/A	N/A	N/A
8bit	MLX Community MLX Community	31.4 GB	N/A	N/A	N/A

Results include trials with 4,096 input tokens and 1,024 output tokens only.

21 devices