- First word afterttft_median_s · s
- 911.69 sper person · median of 4
- Slowest first wordttft_max_s · s
- 1,446.39 sper person · max of 4
- Machine totalaggregate_decode_tps · tok/s
- 0.39 tok/smachine total · single · sample count not recorded
- Machine total including first-word waitaggregate_e2e_tps · tok/s
- 0.29 tok/smachine total · single · sample count not recorded
- Everyone answered inwall_seconds · s
- 1,456.96 smachine total · single · sample count not recorded
- Peak memory, whole hostpeak_host_unavailable_gib · GiB
- 116.28 GiBmachine total · max · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-06 | 5 people · shared · warm | not recorded | 42.02 tok/s | 6.83 s | Passed | |
2026-10-06 | 5 people · shared · cold | not recorded | 32.09 tok/s | 373.25 s | Failed | |
2026-10-06 | 5 people · distinct · cold | not recorded | 2.77 tok/s | 144.94 s | Failed | |
2026-10-06 | 5 people · distinct · cold | not recorded | 1.09 tok/s | 377.19 s | Failed | |
2026-10-06 | 5 people · distinct · warm | not recorded | 0.73 tok/s | 368.02 s | Failed | |
2026-10-06 | 5 people · distinct · cold | not recorded | 0.36 tok/s | 1,090.27 s | Failed | |
2026-10-06 | 4 people · distinct · cold | not recorded | 0.39 tok/s | 911.69 s | Passed | |
2026-10-06 | 3 people · distinct · cold | not recorded | 0.44 tok/s | 733.62 s | Passed | |
2026-10-06 | 2 people · distinct · cold | not recorded | 0.53 tok/s | 546.45 s | Passed | |
2026-10-06 | 1 people · distinct · cold | not recorded | 19.72 tok/s | 361.40 s | Passed | |
2026-10-06 | 1 people · distinct · cold | not recorded | not recorded | not recorded | Failed |
Each person in this run
First-word wait against writing speed after that word, including pauses. Earlier streams can pause while other people’s prompts are read. The table preserves filled input, cached-token counts and each person’s checks.
Per-person table · all measured request rows
| Person | Filled input | Cached tokens | Output | First word | Writing after first word | Total time | Checks | Run |
|---|---|---|---|---|---|---|---|---|
2 | 250,011 | unknown | 103 | 1,446.39 s | 11.10 tok/sone person · includes waits | 1,455.58 s | 3/3 Hidden facts found · completed · checks passed | |
3 | 250,011 | unknown | 106 | 1,091.54 s | 0.29 tok/sone person · includes waits | 1,456.46 s | 3/3 Hidden facts found · completed · checks passed | |
1 | 250,011 | unknown | 108 | 731.84 s | 0.15 tok/sone person · includes waits | 1,456.74 s | 3/3 Hidden facts found · completed · checks passed | |
4 | 250,011 | unknown | 109 | 368.90 s | 0.10 tok/sone person · includes waits | 1,456.95 s | 3/3 Hidden facts found · completed · checks passed |
Per-person table · all measured request rows
| Person | Filled input | Cached tokens | Output | First word | Writing after first word | Total time | Checks | Run |
|---|---|---|---|---|---|---|---|---|
2 | 250,011 | unknown | 103 | 1,446.39 s | 11.10 tok/sone person · includes waits | 1,455.58 s | 3/3 Hidden facts found · completed · checks passed | |
3 | 250,011 | unknown | 106 | 1,091.54 s | 0.29 tok/sone person · includes waits | 1,456.46 s | 3/3 Hidden facts found · completed · checks passed | |
1 | 250,011 | unknown | 108 | 731.84 s | 0.15 tok/sone person · includes waits | 1,456.74 s | 3/3 Hidden facts found · completed · checks passed | |
4 | 250,011 | unknown | 109 | 368.90 s | 0.10 tok/sone person · includes waits | 1,456.95 s | 3/3 Hidden facts found · completed · checks passed |
Answer checks
12/12 Hidden facts found4/4 request checks
Method
Synchronized long public-domain prompts, three exact hidden facts per user. Decode rate uses completion tokens minus one over first output through stream completion; effective prefill includes scheduling and first-token latency. Aggregate decode uses the union window from first output to final completion; end-to-end rate includes prefill.
Limits
Lab observation, not customer acceptance. Prompt content length excludes the server chat template; actual per-request usage is recorded. Synthesis is requested and retained but semantic accuracy is unassessed. All requested users must overlap in observed decode to pass.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
2228b9b22e93a88d84556419c879448ab6c490ae65c4c0b166f4962190ddbf26d3cfb92742e30c8b46564665791dbe0a86ed64cfc02b1275081530793c0c9581db6146a5464fb0a891181b93c81593f0ca65c602eb14120a1c2b1b09bca11f85fbcdb5ba1cdda462b5f38592d071e772c4d398afea61a0aa9188b32d1a239a79- Engine · version · feature
- SGLang ·
0.0.0.dev0+qwen38.27b.g561c8f3· speculative decoding - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-long-context-concurrency-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
1e01cbcbd88ac9878b889cba44f56c74f5ab023f35e2d28a3bbe343923d909e0
Workload shape
- cache state
- cold
- People in this run
- 4
- Configured window (tokens)
- 262144
- Filled content target (tokens)
- 250000
- fixture variant
- distinct-v1
- KV cache precision
- fp8_e4m3
- Output token ceiling
- 512
- scenario
- distinct
- Thinking
- off
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
wall seconds | 1,456.96 s | machine total | single | not recorded | wall_seconds | |
ttft median s | 911.69 s | per person | median | 4 | ttft_median_s | |
hidden facts total | 12.00 count | machine total | single | not recorded | needle_total | |
ttft max s | 1,446.39 s | per person | max | 4 | ttft_max_s | |
requests ok | 4.00 count | machine total | single | not recorded | requests_ok | |
hidden facts correct | 12.00 count | machine total | single | not recorded | needle_correct | |
decode overlap | 4.00 count | machine total | single | not recorded | decode_overlap | |
aggregate concurrency | 4.00 count | machine total | single | not recorded | aggregate_concurrency | |
requests total | 4.00 count | machine total | single | not recorded | requests_total | |
aggregate decode tps | 0.39 tok/s | machine total | single | not recorded | aggregate_tps | |
aggregate e2e tps | 0.29 tok/s | machine total | single | not recorded | aggregate_end_to_end_tps | |
peak host unavailable gib | 116.28 GiB | machine total | max | not recorded | peak_host_unavailable_gib |
Data and citation
Skilak Keep evidence, run 055e55e7006f46ae9559475d, measured 2026-10-06T07:46:35.891108+00:00. https://skilak.ai/keep/resources/evidence/r/055e55e7006f46ae9559475d
