Quant | Actions | ||||||
|---|---|---|---|---|---|---|---|
 mlx_lm0.31.3 | 8.1 tok/s | 343.1 tok/s | 37.00 GB 58% | ||||
| mlx_lm0.31.3 | 11.3 tok/s | 357.5 tok/s | 27.00 GB 42% | ||||
| mlx_lm0.31.3 | 19.1 tok/s | 440.2 tok/s | 23.00 GB 36% | ||||
| mlx_lm0.31.2 | 17.1 tok/s | 586.2 tok/s | 33.00 GB 52% | ||||
| mlx_lm0.31.2 | 3.5 tok/s | 28.3 tok/s | 21.00 GB 66% | ||||
| mlx_lm0.31.2 | 9.3 tok/s | 83.5 tok/s | 37.00 GB 58% | ||||
| mlx_lm0.31.2 | 6.0 tok/s | 49.7 tok/s | 30.00 GB 47% | ||||
| mlx_lm0.31.2 | 7.2 tok/s | 100.7 tok/s | 33.00 GB 52% | ||||
 llama.cppb8680 | 13.0 tok/s | 191.9 tok/s | 10.63 GB 22% | ||||
| mlx_lm0.31.2 | 12.9 tok/s | 351.7 tok/s | 23.00 GB 48% |
Quant | Actions | ||
|---|---|---|---|
8.1 tok/s | |||
11.3 tok/s | |||
19.1 tok/s | |||
17.1 tok/s | |||
3.5 tok/s | |||
9.3 tok/s | |||
6.0 tok/s | |||
7.2 tok/s | |||
13.0 tok/s | |||
12.9 tok/s |