- Code writing speedcode_decode_tps · tok/s
- 33.64 tok/sper person · mean · sample count not recorded
- Prose writing speedchat_decode_tps · tok/s
- 26.99 tok/sper person · mean · sample count not recorded
- Machine totalaggregate_e2e_tps · tok/s
- 53.59 tok/smachine total · single · sample count not recorded
- Peak memory, whole hostpeak_host_unavailable_gib · GiB
- 27.52 GiBmachine total · max · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-05 | 2 people | not recorded | 53.59 tok/s | not recorded | Partial | |
2026-10-05 | 2 people | not recorded | 53.59 tok/s | not recorded | Not graded | |
2026-10-05 | 1 people | not recorded | not recorded | not recorded | Not graded | |
2026-10-05 | unknown people | not recorded | not recorded | not recorded | Failed | |
2026-10-05 | unknown people | not recorded | not recorded | not recorded | Failed |
Answer checks
15/16 tasks
Method
Strict sixteen-task structured-context workload at concurrency 1 and 2, no retries or warmup. Decode, aggregate throughput and memory are from the separate warm code/chat run at concurrency 2; first-answer p95 in the table is from the workload.
Limits
15/16 at both concurrency levels; measured but unqualified. Short prompts and reused caches do not establish long-context quality.
What did not work
- One task returned fenced text instead of one strict JSON value at each concurrency level.
- Native runtime identity does not satisfy the workload runner's OCI identity gate.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- GSQ-RCO IQ3_S GGUF variant · GGUF · GSQ-RCO IQ3_S
- Weights SHA-256
- not recorded
- Engine · version · feature
- llama.cpp ·
native CUDA 13, sm_121a· with multi-token prediction - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
not recorded- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 2
- Configured window (tokens)
- 65536
- Output token ceiling
- 512
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
workload total | 16.00 count | machine total | single | not recorded | workload_total | |
chat decode tps | 26.99 tok/s | per person | mean | not recorded | chat_decode_tps | |
code ttft mean s | 0.30 s | per person | mean | not recorded | code_ttft_mean_s | |
workload passed | 15.00 count | machine total | single | not recorded | workload_passed | |
aggregate concurrency | 2.00 count | machine total | single | not recorded | aggregate_concurrency | |
aggregate e2e tps | 53.59 tok/s | machine total | single | not recorded | aggregate_tps | |
code decode tps | 33.64 tok/s | per person | mean | not recorded | code_decode_tps | |
chat ttft mean s | 0.25 s | per person | mean | not recorded | chat_ttft_mean_s | |
peak host unavailable gib | 27.52 GiB | machine total | max | not recorded | peak_host_unavailable_gib |
Data and citation
Skilak Keep evidence, run d92bd079da7d9fc5722fc394, measured 2026-10-05T18:22:43.041657+00:00. https://skilak.ai/keep/resources/evidence/r/d92bd079da7d9fc5722fc394
