whatcani.run on an M1 Max with 64 GB RAM?
Find the best models and how to run them locally, based on real data from tokens across trials from people.Models
Results include trials with 4,096 input tokens and 1,024 output tokens only.
Actions | ||||||||
|---|---|---|---|---|---|---|---|---|
 llama.cpp | 447.3 tok/s | 10,401.9 tok/s | 394 ms | 0.18 GB 0% | 101 run | Runs great | ||
| llama.cpp | 229.8 tok/s | 10,108.0 tok/s | 386 ms | 0.24 GB 0% | 101 run | Runs great | ||
| llama.cpp | 256.1 tok/s | 10,490.4 tok/s | 391 ms | 0.46 GB 1% | 101 run | Runs great | ||
| llama.cpp | 400.5 tok/s | 9,083.0 tok/s | 441 ms | 0.76 GB 1% | 101 run | Runs great | ||
| llama.cpp | 389.1 tok/s | 7,868.0 tok/s | 503 ms | 0.79 GB 1% | 101 run | Runs great | ||
| llama.cpp | 274.3 tok/s | 10,369.4 tok/s | 392 ms | 1.27 GB 2% | 101 run | Runs great | ||
| llama.cpp | 108.8 tok/s | 4,028.9 tok/s | 1,057 ms | 1.56 GB 2% | 101 run | Runs great | ||
| llama.cpp | 243.2 tok/s | 8,129.5 tok/s | 447 ms | 1.63 GB 3% | 202 runs | Runs great | ||
| llama.cpp | 239.0 tok/s | 8,178.6 tok/s | 472 ms | 1.69 GB 3% | 101 run | Runs great | ||
| llama.cpp | 101.9 tok/s | 2,985.9 tok/s | 1,413 ms | 1.60 GB 3% | 303 runs | Runs great | ||
| llama.cpp | 108.9 tok/s | 2,757.2 tok/s | 1,600 ms | 1.26 GB 2% | 202 runs | Runs great | ||
 mlx_lm | 213.9 tok/s | 2,913.2 tok/s | 1,443 ms | 2.20 GB 3% | 202 runs | Runs great | ||
| mlx_lm | 166.9 tok/s | 2,512.2 tok/s | 1,567 ms | 2.38 GB 4% | 404 runs | Runs great | ||
| llama.cpp | 178.9 tok/s | 2,146.1 tok/s | 2,295 ms | 1.18 GB 2% | 101 run | Runs great | ||
| mlx_lm | 168.6 tok/s | 2,202.7 tok/s | 1,869 ms | 2.56 GB 4% | 727 runs | Runs great | ||
| llama.cpp | 149.5 tok/s | 1,941.9 tok/s | 2,290 ms | 1.41 GB 2% | 101 run | Runs great | ||
| mlx_lm | 154.6 tok/s | 1,991.0 tok/s | 2,240 ms | 2.13 GB 3% | 402 runs | Runs great | ||
| llama.cpp | 89.2 tok/s | 2,098.8 tok/s | 2,327 ms | 2.67 GB 4% | 101 run | Runs great | ||
| mlx_lm | 130.0 tok/s | 1,611.2 tok/s | 2,603 ms | 2.13 GB 3% | 301 run | Runs great | ||
| mlx_lm | 121.4 tok/s | 1,120.7 tok/s | 3,607 ms | 2.76 GB 4% | 101 run | Runs great | ||
| mlx_lm | 113.5 tok/s | 1,053.3 tok/s | 3,662 ms | 2.58 GB 4% | 301 run | Runs great | ||
| mlx_lm | 72.9 tok/s | 2,279.7 tok/s | 1,707 ms | 4.72 GB 7% | 101 run | Runs great | ||
| mlx_lm | 113.3 tok/s | 1,055.3 tok/s | 3,972 ms | 3.05 GB 5% | 101 run | Runs great | ||
| llama.cpp | 75.5 tok/s | 2,137.9 tok/s | 2,106 ms | N/A | 302 runs | Runs great | ||
| mlx_lm | 96.7 tok/s | 1,095.4 tok/s | 3,704 ms | 3.14 GB 5% | 301 run | Runs great |
| Actions | |||
|---|---|---|---|
447.3 tok/s | Runs great | ||
229.8 tok/s | Runs great | ||
256.1 tok/s | Runs great | ||
400.5 tok/s | Runs great | ||
389.1 tok/s | Runs great | ||
274.3 tok/s | Runs great | ||
108.8 tok/s | Runs great | ||
243.2 tok/s | Runs great | ||
239.0 tok/s | Runs great | ||
101.9 tok/s | Runs great | ||
108.9 tok/s | Runs great | ||
213.9 tok/s | Runs great | ||
166.9 tok/s | Runs great | ||
178.9 tok/s | Runs great | ||
168.6 tok/s | Runs great | ||
149.5 tok/s | Runs great | ||
154.6 tok/s | Runs great | ||
89.2 tok/s | Runs great | ||
130.0 tok/s | Runs great | ||
121.4 tok/s | Runs great | ||
113.5 tok/s | Runs great | ||
72.9 tok/s | Runs great | ||
113.3 tok/s | Runs great | ||
75.5 tok/s | Runs great | ||
96.7 tok/s | Runs great |