Quant | Actions | ||||||
|---|---|---|---|---|---|---|---|
 mlx_lm0.31.3 | 51.5 tok/s | 873.3 tok/s | 37.00 GB 77% | ||||
| mlx_lm0.31.3 | 83.4 tok/s | 944.4 tok/s | 21.00 GB 44% | ||||
| mlx_lm0.31.3 | 73.9 tok/s | 752.0 tok/s | 21.00 GB 44% | ||||
| mlx_lm0.31.1 | 70.7 tok/s | 965.9 tok/s | 29.00 GB 45% | ||||
 llama.cppb8920 | 73.5 tok/s | 1,504.0 tok/s | 8.91 GB 14% | ||||
| llama.cppb8920 | 71.5 tok/s | 1,532.6 tok/s | 11.15 GB 17% | ||||
| mlx_lm0.31.3 | 66.7 tok/s | 608.3 tok/s | 21.00 GB 33% | ||||
| mlx_lm0.31.3 | 59.9 tok/s | 872.6 tok/s | 22.00 GB 46% | ||||
| llama.cppb8950 | 22.7 tok/s | 252.9 tok/s | 2.93 GB 18% | ||||
| mlx_lm0.31.2 | 113.0 tok/s | 1,616.7 tok/s | 21.00 GB 16% |
Quant | Actions | ||
|---|---|---|---|
51.5 tok/s | |||
83.4 tok/s | |||
73.9 tok/s | |||
70.7 tok/s | |||
73.5 tok/s | |||
71.5 tok/s | |||
66.7 tok/s | |||
59.9 tok/s | |||
22.7 tok/s | |||
113.0 tok/s |