Answer checks
60/105 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | 20113 | 21.54 s | true · true · recalled | Passed | |
1 · 2 | 40242 | 25.69 s | true · true · recalled | Passed | |
1 · 3 | 60372 | 48.54 s | true · true · recalled | Passed | |
1 · 4 | 80504 | 54.80 s | true · true · recalled | Passed | |
1 · 5 | 100636 | 81.66 s | true · true · recalled | Passed | |
1 · 6 | 120764 | 90.72 s | true · true · recalled | Passed | |
1 · 7 | 140892 | 108.69 s | true · true · recalled | Passed | |
1 · 8 | 161021 | 153.80 s | true · true · recalled | Passed | |
1 · 9 | 181154 | 174.22 s | true · true · recalled | Passed | |
1 · 10 | 201288 | 190.20 s | true · true · recalled | Passed | |
1 · 11 | 221414 | 201.22 s | true · true · recalled | Passed | |
1 · 12 | 241548 | 218.35 s | true · true · recalled | Passed | |
1 · 13 | 181158 | 675.79 s | true · true · incorrect | Failed | |
1 · 14 | 201294 | 209.03 s | true · true · incorrect | Failed | |
1 · 15 | 221419 | 219.36 s | true · true · incorrect | Failed | |
1 · 16 | 241551 | 226.73 s | true · true · incorrect | Failed | |
1 · 17 | 181159 | 879.48 s | true · true · incorrect | Failed | |
1 · 18 | 201288 | 161.03 s | true · true · incorrect | Failed | |
1 · 19 | 221416 | 160.06 s | true · true · incorrect | Failed | |
1 · 20 | 241544 | 113.99 s | true · true · incorrect | Failed | |
1 · 21 | 181142 | 775.66 s | true · true · incorrect | Failed | |
2 · 1 | 20120 | 47.93 s | true · true · recalled | Passed | |
2 · 2 | 40247 | 53.98 s | true · true · recalled | Passed | |
2 · 3 | 60378 | 80.99 s | true · true · recalled | Passed | |
2 · 4 | 80511 | 91.75 s | true · true · recalled | Passed | |
2 · 5 | 100640 | 114.39 s | true · true · recalled | Passed | |
2 · 6 | 120767 | 156.65 s | true · true · recalled | Passed | |
2 · 7 | 140899 | 173.26 s | true · true · recalled | Passed | |
2 · 8 | 161030 | 188.32 s | true · true · recalled | Passed | |
2 · 9 | 181163 | 202.21 s | true · true · recalled | Passed | |
2 · 10 | 201295 | 217.22 s | true · true · recalled | Passed | |
2 · 11 | 221423 | 199.50 s | true · true · recalled | Passed | |
2 · 12 | 241560 | 223.94 s | true · true · recalled | Passed | |
2 · 13 | 181172 | 705.09 s | true · true · incorrect | Failed | |
2 · 14 | 201305 | 230.61 s | true · true · incorrect | Failed | |
2 · 15 | 221436 | 201.81 s | true · true · incorrect | Failed | |
2 · 16 | 241567 | 222.46 s | true · true · incorrect | Failed | |
2 · 17 | 181179 | 674.94 s | true · true · incorrect | Failed | |
2 · 18 | 201309 | 157.07 s | true · true · incorrect | Failed | |
2 · 19 | 221443 | 171.15 s | true · true · incorrect | Failed | |
2 · 20 | 241573 | 194.46 s | true · true · incorrect | Failed | |
2 · 21 | 181175 | 585.70 s | true · true · incorrect | Failed | |
3 · 1 | 20118 | 79.51 s | true · true · recalled | Passed | |
3 · 2 | 40250 | 91.76 s | true · true · recalled | Passed | |
3 · 3 | 60380 | 114.35 s | true · true · recalled | Passed | |
3 · 4 | 80509 | 156.67 s | true · true · recalled | Passed | |
3 · 5 | 100641 | 174.42 s | true · true · recalled | Passed | |
3 · 6 | 120774 | 190.17 s | true · true · recalled | Passed | |
3 · 7 | 140909 | 196.07 s | true · true · recalled | Passed | |
3 · 8 | 161039 | 213.35 s | true · true · recalled | Passed | |
3 · 9 | 181172 | 193.97 s | true · true · recalled | Passed | |
3 · 10 | 201304 | 236.96 s | true · true · recalled | Passed | |
3 · 11 | 221432 | 208.11 s | true · true · recalled | Passed | |
3 · 12 | 241560 | 211.37 s | true · true · recalled | Passed | |
3 · 13 | 181165 | 699.41 s | true · true · incorrect | Failed | |
3 · 14 | 201296 | 216.49 s | true · true · incorrect | Failed | |
3 · 15 | 221425 | 178.86 s | true · true · incorrect | Failed | |
3 · 16 | 241553 | 170.98 s | true · true · incorrect | Failed | |
3 · 17 | 181151 | 658.17 s | true · true · incorrect | Failed | |
3 · 18 | 201282 | 195.72 s | true · true · incorrect | Failed | |
3 · 19 | 221413 | 123.06 s | true · true · incorrect | Failed | |
3 · 20 | 241546 | 131.52 s | true · true · incorrect | Failed | |
3 · 21 | 181157 | 483.53 s | true · true · incorrect | Failed | |
4 · 1 | 20119 | 21.97 s | true · true · recalled | Passed | |
4 · 2 | 40251 | 34.33 s | true · true · recalled | Passed | |
4 · 3 | 60385 | 132.56 s | true · true · recalled | Passed | |
4 · 4 | 80512 | 173.90 s | true · true · recalled | Passed | |
4 · 5 | 100645 | 190.15 s | true · true · recalled | Passed | |
4 · 6 | 120773 | 194.92 s | true · true · recalled | Passed | |
4 · 7 | 140904 | 212.43 s | true · true · recalled | Passed | |
4 · 8 | 161039 | 201.03 s | true · true · recalled | Passed | |
4 · 9 | 181170 | 237.34 s | true · true · recalled | Passed | |
4 · 10 | 201307 | 213.38 s | true · true · recalled | Passed | |
4 · 11 | 221443 | 191.56 s | true · true · recalled | Passed | |
4 · 12 | 241572 | 229.25 s | true · true · recalled | Passed | |
4 · 13 | 181185 | 873.99 s | true · true · incorrect | Failed | |
4 · 14 | 201318 | 116.81 s | true · true · incorrect | Failed | |
4 · 15 | 221454 | 109.41 s | true · true · incorrect | Failed | |
4 · 16 | 241589 | 124.11 s | true · true · incorrect | Failed | |
4 · 17 | 181195 | 771.75 s | true · true · incorrect | Failed | |
4 · 18 | 201331 | 90.80 s | true · true · incorrect | Failed | |
4 · 19 | 221468 | 56.39 s | true · true · incorrect | Failed | |
4 · 20 | 241604 | 106.95 s | true · true · incorrect | Failed | |
4 · 21 | 181200 | 398.97 s | true · true · incorrect | Failed | |
5 · 1 | 20118 | 95.45 s | true · true · recalled | Passed | |
5 · 2 | 40249 | 155.09 s | true · true · recalled | Passed | |
5 · 3 | 60380 | 170.42 s | true · true · recalled | Passed | |
5 · 4 | 80514 | 190.21 s | true · true · recalled | Passed | |
5 · 5 | 100647 | 202.23 s | true · true · recalled | Passed | |
5 · 6 | 120782 | 218.11 s | true · true · recalled | Passed | |
5 · 7 | 140917 | 197.37 s | true · true · recalled | Passed | |
5 · 8 | 161049 | 231.76 s | true · true · recalled | Passed | |
5 · 9 | 181185 | 213.42 s | true · true · recalled | Passed | |
5 · 10 | 201317 | 221.38 s | true · true · recalled | Passed | |
5 · 11 | 221449 | 229.28 s | true · true · recalled | Passed | |
5 · 12 | 241581 | 198.52 s | true · true · recalled | Passed | |
5 · 13 | 181182 | 709.10 s | true · true · incorrect | Failed | |
5 · 14 | 201320 | 157.91 s | true · true · incorrect | Failed | |
5 · 15 | 221457 | 158.76 s | true · true · incorrect | Failed | |
5 · 16 | 241595 | 163.12 s | true · true · incorrect | Failed | |
5 · 17 | 181189 | 604.87 s | true · true · incorrect | Failed | |
5 · 18 | 201320 | 127.62 s | true · true · incorrect | Failed | |
5 · 19 | 221455 | 104.38 s | true · true · incorrect | Failed | |
5 · 20 | 241584 | 111.28 s | true · true · incorrect | Failed | |
5 · 21 | 181188 | 414.86 s | true · true · incorrect | Failed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Lab retrieval and rolling-history evidence only. No summaries. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What did not work
- 60/105 requests passed.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- INT4 variant · safetensors · INT4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 5
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 262144
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
roll ttft max seconds | 879.48 s | per person | max | not recorded | roll_ttft_max_seconds | |
requests ok | 60.00 count | machine total | single | not recorded | requests_ok | |
roll ttft p50 seconds | 675.79 s | per person | median | not recorded | roll_ttft_p50_seconds | |
normal ttft max seconds | 237.34 s | per person | max | not recorded | normal_ttft_max_seconds | |
normal ttft p50 seconds | 173.58 s | per person | median | not recorded | normal_ttft_p50_seconds | |
requests total | 105.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run c4e683141a094897a2b3092b, measured 2026-10-07T11:12:04.658512+00:00. https://skilak.ai/keep/resources/evidence/r/c4e683141a094897a2b3092b
