Meta
Meta
Meta/Llama 3.2 3B Instruct

Runs

View all benchmark runs for this model family.

	Quant						Actions

	Quant		Actions

	Quant						Actions
M4	4bit	mlx_lm0.31.2	30.8 tok/s	305.3 tok/s	3.38 GB 11%
M5 Pro	4bit	mlx_lm0.31.2	106.3 tok/s	3,225.0 tok/s	3.75 GB 8%
M4 Max	4bit	mlx_lm0.31.2	160.1 tok/s	1,548.1 tok/s	3.64 GB 3%
M1 Max	4bit	mlx_lm0.31.0	68.4 tok/s	652.0 tok/s	3.43 GB 5%

	Quant		Actions
M4	4bit	30.8 tok/s
M5 Pro	4bit	106.3 tok/s
M4 Max	4bit	160.1 tok/s
M1 Max	4bit	68.4 tok/s