- Code writing speedcode_decode_tps · tok/s
- 57.53 tok/sper person · mean of 3
- Prose writing speedchat_decode_tps · tok/s
- 34.33 tok/sper person · mean of 3
- Machine totalaggregate_e2e_tps · tok/s
- 56.92 tok/smachine total · single · sample count not recorded
- Peak memory, whole hostpeak_host_unavailable_gib · GiB
- 113.64 GiBmachine total · max · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-05 | 1 people | not recorded | 56.92 tok/s | not recorded | Partial | |
2026-10-05 | unknown people | not recorded | not recorded | not recorded | Failed | |
2026-10-05 | unknown people | not recorded | not recorded | not recorded | Failed | |
2026-10-05 | unknown people | not recorded | not recorded | not recorded | Partial |
Answer checks
15/16 tasks6/6 request checks
Method
Managed package drive with unchanged runtime identity. Sixteen strict structured-context tasks each at concurrency 1 and 2, request seed 17, thinking off. Separate speed smoke: one warmup then three 512-token code/prose requests each at concurrency 1; arithmetic mean of completion tokens divided by first-content-to-DONE time. Aggregate, when present, is a separate warm four-request window at concurrency 2, including TTFT.
Limits
Bounded lab package proof only; configured 262144 context is not long-context quality. Memory is sampled whole-host unavailable memory during speed, not allocator residency or the entire drive peak. Candidate: one concurrency-2 answer returned fenced JSON and failed the unchanged strict contract. Uncalibrated FP8 KV scales remain a quality caveat.
What did not work
- 15/16 tasks passed.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
- not recorded
- Engine · version · feature
- SGLang ·
0.0.0.dev0+qwen38.27b.g561c8f3· speculative decoding - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
not recorded- Workload suite · version
not recorded- Harness SHA-256
446eae1b7d949003bd1379b1d1614c10b4f484a093cea9a3dbab7db56672fe5b- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 1
- Configured window (tokens)
- 262144
- KV cache precision
- fp8_e4m3
- Output token ceiling
- 512
- Feature
- speculative decoding
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
workload total | 16.00 count | machine total | single | not recorded | workload_total | |
chat decode tps | 34.33 tok/s | per person | mean | 3 | chat_decode_tps | |
requests ok | 6.00 count | machine total | single | not recorded | requests_ok | |
code ttft mean s | 0.19 s | per person | mean | 3 | code_ttft_mean_s | |
workload passed | 15.00 count | machine total | single | not recorded | workload_passed | |
aggregate concurrency | 2.00 count | machine total | single | not recorded | aggregate_concurrency | |
requests total | 6.00 count | machine total | single | not recorded | requests_total | |
aggregate e2e tps | 56.92 tok/s | machine total | single | not recorded | aggregate_tps | |
code decode tps | 57.53 tok/s | per person | mean | 3 | code_decode_tps | |
chat ttft mean s | 0.19 s | per person | mean | 3 | chat_ttft_mean_s | |
peak host unavailable gib | 113.64 GiB | machine total | max | not recorded | peak_host_unavailable_gib |
Data and citation
Skilak Keep evidence, run 4d30da78f0b56f97709f43d6, measured 2026-10-05T22:21:12.790316+00:00. https://skilak.ai/keep/resources/evidence/r/4d30da78f0b56f97709f43d6
