Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-07 | 5 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 1 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 2 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 5 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 1 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 1 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 5 people | not recorded | not recorded | not recorded | Failed |
Answer checks
1/2 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | 20229 | 3.96 s | true · true · recalled | Passed | |
1 · 2 | 20288 | 4.04 s | true · true · incorrect | Failed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Bounded tuning pilot; incomplete window growth cannot pass the full workload grade. Lab retrieval and rolling-history evidence only. No summaries. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What did not work
- 1/2 requests passed.
What ran
Exact identity
- Original publisher model
- Qwen3.6-35B-A3B
- Publisher revision
995ad96eacd98c81ed38be0c5b274b04031597b0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 1
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 262144
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
roll ttft max seconds | 4.04 s | per person | max | not recorded | roll_ttft_max_seconds | |
requests ok | 1.00 count | machine total | single | not recorded | requests_ok | |
roll ttft p50 seconds | 4.04 s | per person | median | not recorded | roll_ttft_p50_seconds | |
rolled off abstained | 0.00 count | machine total | single | not recorded | rolled_off_abstained | |
rolled off invalid | 0.00 count | machine total | single | not recorded | rolled_off_invalid | |
rolled off wrong | 1.00 count | machine total | single | not recorded | rolled_off_wrong | |
normal ttft max seconds | 3.96 s | per person | max | not recorded | normal_ttft_max_seconds | |
rolled off correct | 0.00 count | machine total | single | not recorded | rolled_off_correct | |
normal ttft p50 seconds | 3.96 s | per person | median | not recorded | normal_ttft_p50_seconds | |
rolled off total | 1.00 count | machine total | single | not recorded | rolled_off_total | |
requests total | 2.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run 042d9bd08ed94402bc2085e5, measured 2026-10-07T16:31:48.247200+00:00. https://skilak.ai/keep/resources/evidence/r/042d9bd08ed94402bc2085e5
