Google

Google

Google/Gemma 4 12B IT

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q4_0	Unsloth Unsloth	6.3 GB	35.3 tok/s	751.5 tok/s	Runs well	Run
ud-q4_k_xl	Unsloth Unsloth	6.9 GB	32.9 tok/s	619.7 tok/s	Runs ok	Run
OptiQ-4bit	MLX Community MLX Community	8.3 GB	27.5 tok/s	898.2 tok/s	Runs ok	Run
q8_0	Unsloth Unsloth	11.8 GB	21.0 tok/s	620.2 tok/s	Runs ok	Run

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.