Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-07 | 5 people | not recorded | not recorded | not recorded | Passed | |
2026-10-07 | 5 people | not recorded | not recorded | not recorded | Failed | |
2026-10-07 | 1 people | not recorded | not recorded | not recorded | Failed |
Answer checks
105/105 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | 20208 | 21.30 s | true · true · recalled | Passed | |
1 · 2 | 20380 | 21.61 s | true · true · recalled | Passed | |
1 · 3 | 20381 | 34.67 s | true · true · recalled | Passed | |
1 · 4 | 20380 | 27.18 s | true · true · recalled | Passed | |
1 · 5 | 20381 | 36.00 s | true · true · recalled | Passed | |
1 · 6 | 20375 | 38.84 s | true · true · recalled | Passed | |
1 · 7 | 20382 | 40.54 s | true · true · recalled | Passed | |
1 · 8 | 20381 | 61.33 s | true · true · recalled | Passed | |
1 · 9 | 20382 | 49.72 s | true · true · recalled | Passed | |
1 · 10 | 20383 | 50.01 s | true · true · recalled | Passed | |
1 · 11 | 20380 | 51.15 s | true · true · recalled | Passed | |
1 · 12 | 20383 | 66.67 s | true · true · recalled | Passed | |
1 · 13 | 20379 | 70.47 s | true · true · recalled | Passed | |
1 · 14 | 20384 | 79.82 s | true · true · recalled | Passed | |
1 · 15 | 20380 | 67.96 s | true · true · recalled | Passed | |
1 · 16 | 20383 | 63.04 s | true · true · recalled | Passed | |
1 · 17 | 20381 | 52.81 s | true · true · recalled | Passed | |
1 · 18 | 20379 | 51.72 s | true · true · recalled | Passed | |
1 · 19 | 20381 | 57.44 s | true · true · recalled | Passed | |
1 · 20 | 20380 | 58.21 s | true · true · recalled | Passed | |
1 · 21 | 20378 | 52.64 s | true · true · recalled | Passed | |
2 · 1 | 20215 | 36.15 s | true · true · recalled | Passed | |
2 · 2 | 20383 | 28.93 s | true · true · recalled | Passed | |
2 · 3 | 20384 | 45.36 s | true · true · recalled | Passed | |
2 · 4 | 20382 | 40.25 s | true · true · recalled | Passed | |
2 · 5 | 20378 | 41.78 s | true · true · recalled | Passed | |
2 · 6 | 20384 | 61.36 s | true · true · recalled | Passed | |
2 · 7 | 20382 | 51.83 s | true · true · recalled | Passed | |
2 · 8 | 20383 | 48.16 s | true · true · recalled | Passed | |
2 · 9 | 20384 | 52.92 s | true · true · recalled | Passed | |
2 · 10 | 20380 | 69.10 s | true · true · recalled | Passed | |
2 · 11 | 20383 | 70.50 s | true · true · recalled | Passed | |
2 · 12 | 20387 | 83.86 s | true · true · recalled | Passed | |
2 · 13 | 20380 | 65.89 s | true · true · recalled | Passed | |
2 · 14 | 20382 | 22.15 s | true · true · recalled | Passed | |
2 · 15 | 20383 | 25.95 s | true · true · recalled | Passed | |
2 · 16 | 20382 | 32.49 s | true · true · recalled | Passed | |
2 · 17 | 20383 | 22.20 s | true · true · recalled | Passed | |
2 · 18 | 20383 | 23.04 s | true · true · recalled | Passed | |
2 · 19 | 20385 | 30.96 s | true · true · recalled | Passed | |
2 · 20 | 20383 | 42.50 s | true · true · recalled | Passed | |
2 · 21 | 20384 | 42.54 s | true · true · recalled | Passed | |
3 · 1 | 20213 | 32.83 s | true · true · recalled | Passed | |
3 · 2 | 20381 | 21.98 s | true · true · recalled | Passed | |
3 · 3 | 20378 | 22.00 s | true · true · recalled | Passed | |
3 · 4 | 20381 | 42.37 s | true · true · recalled | Passed | |
3 · 5 | 20382 | 25.12 s | true · true · recalled | Passed | |
3 · 6 | 20383 | 34.90 s | true · true · recalled | Passed | |
3 · 7 | 20381 | 29.66 s | true · true · recalled | Passed | |
3 · 8 | 20383 | 41.85 s | true · true · recalled | Passed | |
3 · 9 | 20383 | 46.81 s | true · true · recalled | Passed | |
3 · 10 | 20378 | 79.62 s | true · true · recalled | Passed | |
3 · 11 | 20379 | 71.31 s | true · true · recalled | Passed | |
3 · 12 | 20379 | 63.08 s | true · true · recalled | Passed | |
3 · 13 | 20383 | 54.75 s | true · true · recalled | Passed | |
3 · 14 | 20382 | 51.76 s | true · true · recalled | Passed | |
3 · 15 | 20380 | 59.44 s | true · true · recalled | Passed | |
3 · 16 | 20383 | 55.95 s | true · true · recalled | Passed | |
3 · 17 | 20381 | 54.56 s | true · true · recalled | Passed | |
3 · 18 | 20380 | 42.43 s | true · true · recalled | Passed | |
3 · 19 | 20382 | 42.41 s | true · true · recalled | Passed | |
3 · 20 | 20377 | 42.52 s | true · true · recalled | Passed | |
3 · 21 | 20382 | 40.64 s | true · true · recalled | Passed | |
4 · 1 | 20214 | 58.59 s | true · true · recalled | Passed | |
4 · 2 | 20385 | 43.78 s | true · true · recalled | Passed | |
4 · 3 | 20382 | 46.72 s | true · true · recalled | Passed | |
4 · 4 | 20378 | 45.32 s | true · true · recalled | Passed | |
4 · 5 | 20380 | 68.71 s | true · true · recalled | Passed | |
4 · 6 | 20384 | 68.26 s | true · true · recalled | Passed | |
4 · 7 | 20379 | 82.24 s | true · true · recalled | Passed | |
4 · 8 | 20386 | 75.42 s | true · true · recalled | Passed | |
4 · 9 | 20385 | 63.47 s | true · true · recalled | Passed | |
4 · 10 | 20387 | 51.69 s | true · true · recalled | Passed | |
4 · 11 | 20382 | 51.46 s | true · true · recalled | Passed | |
4 · 12 | 20385 | 56.93 s | true · true · recalled | Passed | |
4 · 13 | 20386 | 53.95 s | true · true · recalled | Passed | |
4 · 14 | 20383 | 47.53 s | true · true · recalled | Passed | |
4 · 15 | 20384 | 42.73 s | true · true · recalled | Passed | |
4 · 16 | 20382 | 42.76 s | true · true · recalled | Passed | |
4 · 17 | 20383 | 43.26 s | true · true · recalled | Passed | |
4 · 18 | 20384 | 41.75 s | true · true · recalled | Passed | |
4 · 19 | 20383 | 22.06 s | true · true · recalled | Passed | |
4 · 20 | 20381 | 21.99 s | true · true · recalled | Passed | |
4 · 21 | 20387 | 22.09 s | true · true · recalled | Passed | |
5 · 1 | 20213 | 42.40 s | true · true · recalled | Passed | |
5 · 2 | 20383 | 24.21 s | true · true · recalled | Passed | |
5 · 3 | 20383 | 25.91 s | true · true · recalled | Passed | |
5 · 4 | 20386 | 29.32 s | true · true · recalled | Passed | |
5 · 5 | 20382 | 22.15 s | true · true · recalled | Passed | |
5 · 6 | 20385 | 22.26 s | true · true · recalled | Passed | |
5 · 7 | 20381 | 22.18 s | true · true · recalled | Passed | |
5 · 8 | 20386 | 22.29 s | true · true · recalled | Passed | |
5 · 9 | 20383 | 22.22 s | true · true · recalled | Passed | |
5 · 10 | 20382 | 22.15 s | true · true · recalled | Passed | |
5 · 11 | 20381 | 22.16 s | true · true · recalled | Passed | |
5 · 12 | 20382 | 22.22 s | true · true · recalled | Passed | |
5 · 13 | 20383 | 24.17 s | true · true · recalled | Passed | |
5 · 14 | 20382 | 30.92 s | true · true · recalled | Passed | |
5 · 15 | 20385 | 42.89 s | true · true · recalled | Passed | |
5 · 16 | 20388 | 43.08 s | true · true · recalled | Passed | |
5 · 17 | 20387 | 22.21 s | true · true · recalled | Passed | |
5 · 18 | 20386 | 22.21 s | true · true · recalled | Passed | |
5 · 19 | 20384 | 22.02 s | true · true · recalled | Passed | |
5 · 20 | 20383 | 21.93 s | true · true · recalled | Passed | |
5 · 21 | 20383 | 22.02 s | true · true · recalled | Passed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Lab retrieval and rolling-history evidence only. Bounded literal user-source excerpts, with verified source hashes; details outside them remain unavailable. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- INT4 variant · safetensors · INT4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 5
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 262144
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
roll ttft max seconds | 83.86 s | per person | max | not recorded | roll_ttft_max_seconds | |
requests ok | 105.00 count | machine total | single | not recorded | requests_ok | |
roll ttft p50 seconds | 42.64 s | per person | median | not recorded | roll_ttft_p50_seconds | |
rolled off abstained | 0.00 count | machine total | single | not recorded | rolled_off_abstained | |
rolled off invalid | 0.00 count | machine total | single | not recorded | rolled_off_invalid | |
rolled off wrong | 0.00 count | machine total | single | not recorded | rolled_off_wrong | |
normal ttft max seconds | 58.59 s | per person | max | not recorded | normal_ttft_max_seconds | |
rolled off correct | 100.00 count | machine total | single | not recorded | rolled_off_correct | |
normal ttft p50 seconds | 36.15 s | per person | median | not recorded | normal_ttft_p50_seconds | |
rolled off total | 100.00 count | machine total | single | not recorded | rolled_off_total | |
requests total | 105.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run 861767d1989c495e851fd421, measured 2026-10-07T22:28:13.809032+00:00. https://skilak.ai/keep/resources/evidence/r/861767d1989c495e851fd421
