Quant | Actions | ||||||
|---|---|---|---|---|---|---|---|
 llama.cppb8920 | 32.9 tok/s | 619.7 tok/s | 6.05 GB 9% | ||||
| llama.cppb8920 | 21.0 tok/s | 620.2 tok/s | 7.46 GB 12% | ||||
| llama.cppb8920 | 35.3 tok/s | 751.5 tok/s | 4.62 GB 7% | ||||
| llama.cppb9430 | 12.2 tok/s | 126.7 tok/s | 4.40 GB 18% | ||||
| llama.cppb9430 | 13.4 tok/s | 136.0 tok/s | 7.49 GB 31% | ||||
| llama.cppb9430 | 7.4 tok/s | 137.3 tok/s | 7.28 GB 30% | ||||
 mlx_lm0.31.3 | 27.5 tok/s | 898.2 tok/s | 12.00 GB 19% |
Quant | Actions | ||
|---|---|---|---|
32.9 tok/s | |||
21.0 tok/s | |||
35.3 tok/s | |||
12.2 tok/s | |||
13.4 tok/s | |||
7.4 tok/s | |||
27.5 tok/s |