Answer checks
49/50 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | 20208 | 3.97 s | true · true · recalled | Passed | |
1 · 2 | 40330 | 5.73 s | true · true · recalled | Passed | |
1 · 3 | 60454 | 7.00 s | true · true · recalled | Passed | |
1 · 4 | 80583 | 8.79 s | true · true · recalled | Passed | |
1 · 5 | 100707 | 10.51 s | true · true · recalled | Passed | |
1 · 6 | 120833 | 11.05 s | true · true · recalled | Passed | |
1 · 7 | 80753 | 27.63 s | true · true · recalled | Passed | |
1 · 8 | 100875 | 10.67 s | true · true · recalled | Passed | |
1 · 9 | 121001 | 16.26 s | true · true · recalled | Passed | |
1 · 10 | 80759 | 32.22 s | true · true · recalled | Passed | |
2 · 1 | 20214 | 8.94 s | true · true · recalled | Passed | |
2 · 2 | 40334 | 6.66 s | true · true · recalled | Passed | |
2 · 3 | 60457 | 15.63 s | true · true · recalled | Passed | |
2 · 4 | 80585 | 11.85 s | true · true · recalled | Passed | |
2 · 5 | 100712 | 10.51 s | true · true · recalled | Passed | |
2 · 6 | 120832 | 11.06 s | true · true · recalled | Passed | |
2 · 7 | 80760 | 27.87 s | true · true · recalled | Passed | |
2 · 8 | 100884 | 10.60 s | true · true · recalled | Passed | |
2 · 9 | 121014 | 17.19 s | true · true · recalled | Passed | |
2 · 10 | 80765 | 33.59 s | true · true · recalled | Passed | |
3 · 1 | 20209 | 8.31 s | true · true · recalled | Passed | |
3 · 2 | 40335 | 6.02 s | true · true · recalled | Passed | |
3 · 3 | 60463 | 7.34 s | true · true · recalled | Passed | |
3 · 4 | 80585 | 17.83 s | true · true · recalled | Passed | |
3 · 5 | 100714 | 13.32 s | true · true · recalled | Passed | |
3 · 6 | 120840 | 11.46 s | true · true · recalled | Passed | |
3 · 7 | 80763 | 24.56 s | true · true · recalled | Passed | |
3 · 8 | 100887 | 10.17 s | true · true · recalled | Passed | |
3 · 9 | 121013 | 11.18 s | true · true · recalled | Passed | |
3 · 10 | 80758 | 27.71 s | true · true · recalled | Passed | |
4 · 1 | 20212 | 10.22 s | true · true · recalled | Passed | |
4 · 2 | 40337 | 5.81 s | true · true · recalled | Passed | |
4 · 3 | 60470 | 7.02 s | true · true · recalled | Passed | |
4 · 4 | 80594 | 8.85 s | true · true · recalled | Passed | |
4 · 5 | 100719 | 10.10 s | true · true · recalled | Passed | |
4 · 6 | 120840 | 12.66 s | true · true · recalled | Passed | |
4 · 7 | 80755 | 29.36 s | true · true · recalled | Passed | |
4 · 8 | 100888 | 10.62 s | true · true · recalled | Passed | |
4 · 9 | 121013 | 18.54 s | true · true · recalled | Passed | |
4 · 10 | 80770 | 34.95 s | true · true · recalled | Passed | |
5 · 1 | 20214 | 10.35 s | true · true · recalled | Passed | |
5 · 2 | 40338 | 9.02 s | true · true · incorrect | Failed | |
5 · 3 | 60464 | 7.32 s | true · true · recalled | Passed | |
5 · 4 | 80590 | 9.24 s | true · true · recalled | Passed | |
5 · 5 | 100716 | 19.11 s | true · true · recalled | Passed | |
5 · 6 | 120844 | 14.17 s | true · true · recalled | Passed | |
5 · 7 | 80765 | 24.99 s | true · true · recalled | Passed | |
5 · 8 | 100891 | 10.18 s | true · true · recalled | Passed | |
5 · 9 | 121017 | 11.18 s | true · true · recalled | Passed | |
5 · 10 | 80766 | 24.52 s | true · true · recalled | Passed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Bounded tuning pilot; incomplete window growth cannot pass the full workload grade. Lab retrieval and rolling-history evidence only. Bounded literal user-source excerpts, with verified source hashes; details outside them remain unavailable. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What did not work
- 49/50 requests passed.
What ran
Exact identity
- Original publisher model
- Qwen3.6-35B-A3B
- Publisher revision
995ad96eacd98c81ed38be0c5b274b04031597b0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 5
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 262144
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
roll ttft max seconds | 34.95 s | per person | max | not recorded | roll_ttft_max_seconds | |
requests ok | 49.00 count | machine total | single | not recorded | requests_ok | |
roll ttft p50 seconds | 27.79 s | per person | median | not recorded | roll_ttft_p50_seconds | |
rolled off abstained | 0.00 count | machine total | single | not recorded | rolled_off_abstained | |
rolled off invalid | 0.00 count | machine total | single | not recorded | rolled_off_invalid | |
rolled off wrong | 0.00 count | machine total | single | not recorded | rolled_off_wrong | |
normal ttft max seconds | 19.11 s | per person | max | not recorded | normal_ttft_max_seconds | |
rolled off correct | 20.00 count | machine total | single | not recorded | rolled_off_correct | |
normal ttft p50 seconds | 10.43 s | per person | median | not recorded | normal_ttft_p50_seconds | |
rolled off total | 20.00 count | machine total | single | not recorded | rolled_off_total | |
requests total | 50.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run 7cde518077a3402d96c45d36, measured 2026-10-07T19:26:02.104296+00:00. https://skilak.ai/keep/resources/evidence/r/7cde518077a3402d96c45d36
