- Prose writing speedchat_decode_tps · tok/s
- 25.08 tok/sper person · mean · sample count not recorded
- Writing speed, one personsingle_user_decode_tps · tok/s
- 25.08 tok/sper person · single · sample count not recorded
- First word afterttft_median_s · s
- 6.50 sper person · median · sample count not recorded
- Slowest first wordttft_max_s · s
- 6.60 sper person · max · sample count not recorded
- Machine totalaggregate_e2e_tps · tok/s
- 25.08 tok/smachine total · single · sample count not recorded
- Peak memory, whole hostpeak_host_unavailable_gib · GiB
- 14.34 GiBmachine total · max · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-09 | 2 people | not recorded | 38.82 tok/s | 11.27 s | Not graded | |
2026-10-09 | 1 people | not recorded | 25.08 tok/s | 6.50 s | Not graded | |
2026-10-09 | 1 people | not recorded | 25.08 tok/s | not recorded | Not graded | |
2026-10-09 | 2 people | not recorded | 38.82 tok/s | not recorded | Not graded |
Answer checks
3/3 request checks
Method
Code or prose speed smoke, three measured batches after one excluded warmup per user count. Returned token accounting; per-person decode and simultaneous aggregate decode are separate.
Limits
Speed only; outputs reach the token ceiling and do not establish correctness. Whole-Mac memory includes existing applications. Filled contexts observed whole-Mac swap growth; a memory-stable recipe, managed lifecycle and delivery remain unassessed.
What ran
Exact identity
- Original publisher model
- Qwen3-8B-Base
- Publisher revision
49e3418fbbbca6ecbdf9608b4d22e5a407081db4- Variant · format · precision
- Q4_K_M GGUF variant · GGUF · Q4_K_M
- Weights SHA-256
- not recorded
- Engine · version · feature
- llama.cpp Metal ·
b11182· plain - Hardware
- Apple M6 Mac mini, 16 GiB unified memory
- Harness / workload
not recorded- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 1
- Configured window (tokens)
- 5120
- KV cache precision
- q8_0
- Output token ceiling
- 512
- Thinking
- off
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
ttft median s | 6.50 s | per person | median | not recorded | ttft_median_s | |
chat decode tps | 25.08 tok/s | per person | mean | not recorded | chat_decode_tps | |
ttft max s | 6.60 s | per person | max | not recorded | ttft_max_s | |
requests ok | 3.00 count | machine total | single | not recorded | requests_ok | |
aggregate concurrency | 1.00 count | machine total | single | not recorded | aggregate_concurrency | |
requests total | 3.00 count | machine total | single | not recorded | requests_total | |
aggregate e2e tps | 25.08 tok/s | machine total | single | not recorded | aggregate_tps | |
single user decode tps | 25.08 tok/s | per person | single | not recorded | single_user_decode_tps | |
chat ttft mean s | 6.53 s | per person | mean | not recorded | chat_ttft_mean_s | |
peak host unavailable gib | 14.34 GiB | machine total | max | not recorded | peak_host_unavailable_gib |
Data and citation
Skilak Keep evidence, run 21a91b3062bf439f871434e3, measured 2026-10-09T20:18:54.084891+00:00. https://skilak.ai/keep/resources/evidence/r/21a91b3062bf439f871434e3
