Answer checks
59/105 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | 20113 | 4.62 s | true · true · recalled | Passed | |
1 · 2 | 40238 | 5.75 s | true · true · recalled | Passed | |
1 · 3 | 60364 | 6.94 s | true · true · recalled | Passed | |
1 · 4 | 80492 | 8.79 s | true · true · recalled | Passed | |
1 · 5 | 100620 | 10.00 s | true · true · recalled | Passed | |
1 · 6 | 120744 | 10.97 s | true · true · recalled | Passed | |
1 · 7 | 140868 | 15.05 s | true · true · recalled | Passed | |
1 · 8 | 160993 | 18.29 s | true · true · recalled | Passed | |
1 · 9 | 181122 | 21.94 s | true · true · recalled | Passed | |
1 · 10 | 201252 | 24.99 s | true · true · recalled | Passed | |
1 · 11 | 221374 | 26.37 s | true · true · recalled | Passed | |
1 · 12 | 241504 | 32.42 s | true · true · recalled | Passed | |
1 · 13 | 181126 | 119.66 s | true · true · incorrect | Failed | |
1 · 14 | 201257 | 21.21 s | true · true · incorrect | Failed | |
1 · 15 | 221379 | 35.19 s | true · true · incorrect | Failed | |
1 · 16 | 241508 | 59.34 s | true · true · incorrect | Failed | |
1 · 17 | 181129 | 216.08 s | true · true · incorrect | Failed | |
1 · 18 | 201255 | 33.66 s | true · true · incorrect | Failed | |
1 · 19 | 221380 | 65.85 s | true · true · incorrect | Failed | |
1 · 20 | 241505 | 67.60 s | true · true · incorrect | Failed | |
1 · 21 | 181116 | 275.56 s | true · true · incorrect | Failed | |
2 · 1 | 20120 | 11.53 s | true · true · recalled | Passed | |
2 · 2 | 40243 | 18.30 s | true · true · incorrect | Failed | |
2 · 3 | 60370 | 21.92 s | true · true · recalled | Passed | |
2 · 4 | 80499 | 24.55 s | true · true · recalled | Passed | |
2 · 5 | 100624 | 26.52 s | true · true · recalled | Passed | |
2 · 6 | 120747 | 29.09 s | true · true · recalled | Passed | |
2 · 7 | 140875 | 19.78 s | true · true · recalled | Passed | |
2 · 8 | 161002 | 30.13 s | true · true · recalled | Passed | |
2 · 9 | 181131 | 24.17 s | true · true · recalled | Passed | |
2 · 10 | 201259 | 25.84 s | true · true · recalled | Passed | |
2 · 11 | 221383 | 41.40 s | true · true · recalled | Passed | |
2 · 12 | 241516 | 60.02 s | true · true · recalled | Passed | |
2 · 13 | 181140 | 215.31 s | true · true · incorrect | Failed | |
2 · 14 | 201268 | 33.46 s | true · true · incorrect | Failed | |
2 · 15 | 221391 | 63.18 s | true · true · incorrect | Failed | |
2 · 16 | 241517 | 66.50 s | true · true · incorrect | Failed | |
2 · 17 | 181141 | 276.94 s | true · true · incorrect | Failed | |
2 · 18 | 201270 | 65.36 s | true · true · incorrect | Failed | |
2 · 19 | 221403 | 55.90 s | true · true · incorrect | Failed | |
2 · 20 | 241531 | 40.94 s | true · true · incorrect | Failed | |
2 · 21 | 181148 | 228.88 s | true · true · incorrect | Failed | |
3 · 1 | 20118 | 27.37 s | true · true · recalled | Passed | |
3 · 2 | 40246 | 20.19 s | true · true · recalled | Passed | |
3 · 3 | 60372 | 29.64 s | true · true · recalled | Passed | |
3 · 4 | 80497 | 23.42 s | true · true · recalled | Passed | |
3 · 5 | 100625 | 25.99 s | true · true · recalled | Passed | |
3 · 6 | 120754 | 40.51 s | true · true · recalled | Passed | |
3 · 7 | 140885 | 60.09 s | true · true · recalled | Passed | |
3 · 8 | 161011 | 33.26 s | true · true · recalled | Passed | |
3 · 9 | 181140 | 53.18 s | true · true · recalled | Passed | |
3 · 10 | 201268 | 35.08 s | true · true · recalled | Passed | |
3 · 11 | 221392 | 39.30 s | true · true · recalled | Passed | |
3 · 12 | 241516 | 69.03 s | true · true · recalled | Passed | |
3 · 13 | 181133 | 215.44 s | true · true · incorrect | Failed | |
3 · 14 | 201261 | 46.06 s | true · true · incorrect | Failed | |
3 · 15 | 221389 | 64.95 s | true · true · incorrect | Failed | |
3 · 16 | 241516 | 57.47 s | true · true · incorrect | Failed | |
3 · 17 | 181128 | 251.12 s | true · true · incorrect | Failed | |
3 · 18 | 201258 | 62.15 s | true · true · incorrect | Failed | |
3 · 19 | 221387 | 48.99 s | true · true · incorrect | Failed | |
3 · 20 | 241516 | 40.72 s | true · true · incorrect | Failed | |
3 · 21 | 181142 | 211.02 s | true · true · incorrect | Failed | |
4 · 1 | 20119 | 17.67 s | true · true · recalled | Passed | |
4 · 2 | 40247 | 39.03 s | true · true · recalled | Passed | |
4 · 3 | 60377 | 59.30 s | true · true · recalled | Passed | |
4 · 4 | 80500 | 32.47 s | true · true · recalled | Passed | |
4 · 5 | 100629 | 52.05 s | true · true · recalled | Passed | |
4 · 6 | 120753 | 31.70 s | true · true · recalled | Passed | |
4 · 7 | 140880 | 39.83 s | true · true · recalled | Passed | |
4 · 8 | 161011 | 63.29 s | true · true · recalled | Passed | |
4 · 9 | 181138 | 66.80 s | true · true · recalled | Passed | |
4 · 10 | 201271 | 51.48 s | true · true · recalled | Passed | |
4 · 11 | 221403 | 52.65 s | true · true · recalled | Passed | |
4 · 12 | 241528 | 67.30 s | true · true · recalled | Passed | |
4 · 13 | 181153 | 192.69 s | true · true · incorrect | Failed | |
4 · 14 | 201282 | 51.60 s | true · true · incorrect | Failed | |
4 · 15 | 221413 | 59.58 s | true · true · incorrect | Failed | |
4 · 16 | 241544 | 52.31 s | true · true · incorrect | Failed | |
4 · 17 | 181165 | 155.46 s | true · true · incorrect | Failed | |
4 · 18 | 201296 | 44.99 s | true · true · incorrect | Failed | |
4 · 19 | 221428 | 32.99 s | true · true · incorrect | Failed | |
4 · 20 | 241559 | 35.60 s | true · true · incorrect | Failed | |
4 · 21 | 181168 | 121.23 s | true · true · incorrect | Failed | |
5 · 1 | 20118 | 37.70 s | true · true · recalled | Passed | |
5 · 2 | 40245 | 60.05 s | true · true · recalled | Passed | |
5 · 3 | 60372 | 33.31 s | true · true · recalled | Passed | |
5 · 4 | 80502 | 52.87 s | true · true · recalled | Passed | |
5 · 5 | 100631 | 34.96 s | true · true · recalled | Passed | |
5 · 6 | 120762 | 39.33 s | true · true · recalled | Passed | |
5 · 7 | 140893 | 56.77 s | true · true · recalled | Passed | |
5 · 8 | 161021 | 66.22 s | true · true · recalled | Passed | |
5 · 9 | 181153 | 51.70 s | true · true · recalled | Passed | |
5 · 10 | 201281 | 54.78 s | true · true · recalled | Passed | |
5 · 11 | 221409 | 65.36 s | true · true · recalled | Passed | |
5 · 12 | 241537 | 56.99 s | true · true · recalled | Passed | |
5 · 13 | 181150 | 246.41 s | true · true · incorrect | Failed | |
5 · 14 | 201284 | 62.80 s | true · true · incorrect | Failed | |
5 · 15 | 221417 | 51.88 s | true · true · incorrect | Failed | |
5 · 16 | 241553 | 41.32 s | true · true · incorrect | Failed | |
5 · 17 | 181163 | 212.36 s | true · true · incorrect | Failed | |
5 · 18 | 201292 | 33.33 s | true · true · incorrect | Failed | |
5 · 19 | 221424 | 35.20 s | true · true · incorrect | Failed | |
5 · 20 | 241552 | 24.32 s | true · true · incorrect | Failed | |
5 · 21 | 181169 | 128.96 s | true · true · incorrect | Failed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Lab retrieval and rolling-history evidence only. No summaries. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What did not work
- 59/105 requests passed.
What ran
Exact identity
- Original publisher model
- Qwen3.6-35B-A3B
- Publisher revision
995ad96eacd98c81ed38be0c5b274b04031597b0- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 5
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 262144
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
roll ttft max seconds | 276.94 s | per person | max | not recorded | roll_ttft_max_seconds | |
requests ok | 59.00 count | machine total | single | not recorded | requests_ok | |
roll ttft p50 seconds | 215.31 s | per person | median | not recorded | roll_ttft_p50_seconds | |
normal ttft max seconds | 69.03 s | per person | max | not recorded | normal_ttft_max_seconds | |
normal ttft p50 seconds | 39.16 s | per person | median | not recorded | normal_ttft_p50_seconds | |
requests total | 105.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run 5f1b8f8e534b4032a6bb6205, measured 2026-10-07T08:41:29.987232+00:00. https://skilak.ai/keep/resources/evidence/r/5f1b8f8e534b4032a6bb6205
