Jackrong

Jackrong

Jackrong/Gemopus 4 E4B IT

Quantizations

Quant	Quantized by	Size	Decode	Prefill	Score	Actions
q6_k	Jackrong Jackrong	5.8 GB	63.4 tok/s	1,123.8 tok/s	Runs well	Run
q8_0	Jackrong Jackrong	7.5 GB	54.4 tok/s	1,180.2 tok/s	Runs well	Run
bf16	Jackrong Jackrong	14.0 GB	33.5 tok/s	1,161.9 tok/s	Runs ok	Run

Device Comparison

Results include trials with 4,096 input tokens and 1,024 output tokens only.