Qwen

Qwen

Qwen/Qwen3.5-9B

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q4_k_s	Unsloth Unsloth	5.0 GB	N/A	N/A	N/A	Run
q4_k_m	Unsloth Unsloth	5.3 GB	N/A	N/A	N/A	Run
4bit	MLX Community MLX Community	5.5 GB	65.9 tok/s	628.9 tok/s	Runs well	Run
OptiQ-4bit	MLX Community MLX Community	5.6 GB	60.9 tok/s	707.4 tok/s	Runs well	Run
q8_0	Unknown	8.9 GB	N/A	N/A	N/A	Run
q8_0	Unsloth Unsloth	8.9 GB	35.9 tok/s	696.6 tok/s	Runs ok	Run
bf16	Unsloth Unsloth	16.7 GB	21.3 tok/s	707.1 tok/s	Runs poorly	Run
bf16	MLX Community MLX Community	17.5 GB	21.9 tok/s	754.4 tok/s	Runs poorly	Run

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.