Dan Lougen
Dan Lougen/Ornstein-27B

OverviewRuns

Quantizations

QuantQuantized bySizeDecodePrefillScoreActions
q4_k_mGGML's logo.
Dan Lougen
Dan Lougen		15.4 GB12.1 tok/s120.7 tok/sRuns poorly

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.

Decode / Prefill Speeds

1 device