- First word afterttft_median_s · s
- 14.47 sper person · median of 5
- Slowest first wordttft_max_s · s
- 14.47 sper person · max of 5
- Machine totalaggregate_decode_tps · tok/s
- 79.96 tok/smachine total · single · sample count not recorded
- Machine total including first-word waitaggregate_e2e_tps · tok/s
- 30.89 tok/smachine total · single · sample count not recorded
- Everyone answered inwall_seconds · s
- 17.65 smachine total · single · sample count not recorded
- Peak memory, whole hostpeak_host_unavailable_gib · GiB
- 93.87 GiBmachine total · max · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-07 | 5 people · shared · warm | not recorded | 79.96 tok/s | 14.47 s | Failed | |
2026-10-07 | 5 people · shared · cold | not recorded | 58.10 tok/s | 676.94 s | Failed | |
2026-10-07 | 5 people · distinct · cold | not recorded | 15.55 tok/s | 676.47 s | Passed |
Each person in this run
First-word wait against writing speed after that word, including pauses. Earlier streams can pause while other people’s prompts are read. The table preserves filled input, cached-token counts and each person’s checks.
Per-person table · all measured request rows
| Person | Filled input | Cached tokens | Output | First word | Writing after first word | Total time | Checks | Run |
|---|---|---|---|---|---|---|---|---|
2 | 250,011 | unknown | 99 | 14.47 s | 35.74 tok/sone person · includes waits | 17.21 s | 2/3 Hidden facts found · completed · checks missed | |
3 | 250,011 | unknown | 104 | 14.47 s | 36.62 tok/sone person · includes waits | 17.29 s | 3/3 Hidden facts found · completed · checks passed | |
4 | 250,011 | unknown | 108 | 10.88 s | 16.57 tok/sone person · includes waits | 17.34 s | 2/3 Hidden facts found · completed · checks missed | |
1 | 250,011 | unknown | 118 | 13.26 s | 27.58 tok/sone person · includes waits | 17.50 s | 3/3 Hidden facts found · completed · checks passed | |
5 | 250,011 | unknown | 116 | 14.47 s | 36.35 tok/sone person · includes waits | 17.64 s | 3/3 Hidden facts found · completed · checks passed |
Per-person table · all measured request rows
| Person | Filled input | Cached tokens | Output | First word | Writing after first word | Total time | Checks | Run |
|---|---|---|---|---|---|---|---|---|
2 | 250,011 | unknown | 99 | 14.47 s | 35.74 tok/sone person · includes waits | 17.21 s | 2/3 Hidden facts found · completed · checks missed | |
3 | 250,011 | unknown | 104 | 14.47 s | 36.62 tok/sone person · includes waits | 17.29 s | 3/3 Hidden facts found · completed · checks passed | |
4 | 250,011 | unknown | 108 | 10.88 s | 16.57 tok/sone person · includes waits | 17.34 s | 2/3 Hidden facts found · completed · checks missed | |
1 | 250,011 | unknown | 118 | 13.26 s | 27.58 tok/sone person · includes waits | 17.50 s | 3/3 Hidden facts found · completed · checks passed | |
5 | 250,011 | unknown | 116 | 14.47 s | 36.35 tok/sone person · includes waits | 17.64 s | 3/3 Hidden facts found · completed · checks passed |
Answer checks
13/15 Hidden facts found3/5 request checks
Method
Synchronized long public-domain prompts, three exact hidden facts per user. Decode rate uses completion tokens minus one over first output through stream completion; effective prefill includes scheduling and first-token latency. Aggregate decode uses the union window from first output to final completion; end-to-end rate includes prefill.
Limits
Lab observation, not customer acceptance. Prompt content length excludes the server chat template; actual per-request usage is recorded. Synthesis is requested and retained but semantic accuracy is unassessed. All requested users must overlap in observed decode to pass.
What did not work
- One or more requests did not meet completion, exact retrieval, synthesis-presence, reasoning-off or token-accounting requirements.
What ran
Exact identity
- Original publisher model
- Qwen3.6-35B-A3B
- Publisher revision
995ad96eacd98c81ed38be0c5b274b04031597b0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
07141c2db92e47bc08777132cd1a0323faf300eab3a7d7c111bc2bf075fda0506dea9c759a0f941cf9e1cc1501216b0e107966a66c90425050e212efbd053f029758875fc55e49561165f4a44342b654c123c3e25c6811b34abb83553fb1a164- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· with multi-token prediction - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-long-context-concurrency-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
77fda79ee1961c35a8f8a77e7d5a01d5f9ce414975e00e640a166e71e5f9093e
Workload shape
- cache state
- warm
- People in this run
- 5
- Configured window (tokens)
- 262144
- Filled content target (tokens)
- 250000
- fixture variant
- shared-record-groups-v2
- KV cache precision
- fp8
- Output token ceiling
- 512
- scenario
- shared
- Feature
- with multi-token prediction
- Thinking
- off
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
wall seconds | 17.65 s | machine total | single | not recorded | wall_seconds | |
ttft median s | 14.47 s | per person | median | 5 | ttft_median_s | |
hidden facts total | 15.00 count | machine total | single | not recorded | needle_total | |
ttft max s | 14.47 s | per person | max | 5 | ttft_max_s | |
requests ok | 3.00 count | machine total | single | not recorded | requests_ok | |
hidden facts correct | 13.00 count | machine total | single | not recorded | needle_correct | |
latency bar seconds | 300.00 s | machine total | single | not recorded | latency_bar_seconds | |
decode overlap | 5.00 count | machine total | single | not recorded | decode_overlap | |
aggregate concurrency | 5.00 count | machine total | single | not recorded | aggregate_concurrency | |
requests total | 5.00 count | machine total | single | not recorded | requests_total | |
aggregate decode tps | 79.96 tok/s | machine total | single | not recorded | aggregate_tps | |
aggregate e2e tps | 30.89 tok/s | machine total | single | not recorded | aggregate_end_to_end_tps | |
peak host unavailable gib | 93.87 GiB | machine total | max | not recorded | peak_host_unavailable_gib |
Data and citation
Skilak Keep evidence, run 68146b5a341248dd966978ee, measured 2026-10-07T05:25:56.863392+00:00. https://skilak.ai/keep/resources/evidence/r/68146b5a341248dd966978ee
