Quant | Actions | ||||||
|---|---|---|---|---|---|---|---|
 mlx_lm0.31.2 | 77.3 tok/s | 1,012.2 tok/s | 45.00 GB 35% | ||||
| mlx_lm0.31.2 | 47.1 tok/s | 648.5 tok/s | 45.00 GB 70% | ||||
| mlx_lm0.31.2 | 80.4 tok/s | 1,311.4 tok/s | 45.00 GB 35% | ||||
| mlx_lm0.31.2 | 65.3 tok/s | 573.3 tok/s | 43.00 GB 67% | ||||
 llama.cppb8793 | 39.4 tok/s | 469.7 tok/s | 28.74 GB 45% |
Quant | Actions | ||
|---|---|---|---|
77.3 tok/s | |||
47.1 tok/s | |||
80.4 tok/s | |||
65.3 tok/s | |||
39.4 tok/s |