Qwen

Qwen

Qwen/Qwen3.5-4B

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q4_k_s	Unsloth Unsloth	2.4 GB	N/A	N/A	N/A	Run
q4_k_m	Unsloth Unsloth	2.6 GB	52.2 tok/s	777.3 tok/s	Runs well	Run
4bit	MLX Community MLX Community	2.8 GB	82.7 tok/s	575.2 tok/s	Runs well	Run
q8_0	Unknown	4.2 GB	N/A	N/A	N/A	N/A

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.