Qwen
Qwen
Qwen/Qwen3.5-0.8B

Runs

View all benchmark runs for this model family.

	Quant
M4 Max	OptiQ-4bit	mlx_lm0.31.2	363.5 tok/s	8,625.9 tok/s	2.59 GB 2%
M4	OptiQ-4bit	mlx_lm0.31.2	127.9 tok/s	2,290.9 tok/s	2.30 GB 14%
GeForce RTX 5070 Ti	q4_k_m	llama.cppb1	374.9 tok/s	21,960.3 tok/s	0.86 GB 4%
M2 Pro	8bit	mlx_lm0.31.1	150.2 tok/s	1,877.0 tok/s	2.60 GB 16%
M2 Pro	q8_0	llama.cppb8680	73.4 tok/s	2,748.4 tok/s	1.42 GB 9%
M2 Pro	q4_k_m	llama.cppb8680	101.1 tok/s	2,389.9 tok/s	1.92 GB 12%
M2 Pro	q3_k_m	llama.cppb8680	97.9 tok/s	2,500.9 tok/s	0.75 GB 5%
M2 Pro	ud-iq2_xxs	llama.cppb8680	99.2 tok/s	2,325.4 tok/s	1.01 GB 6%
M4 Max	OptiQ-4bit	mlx_lm0.31.1	326.9 tok/s	7,137.5 tok/s	2.55 GB 7%
M3 Max	OptiQ-4bit	mlx_lm0.31.1	313.4 tok/s	7,438.0 tok/s	2.62 GB 2%