Answer checks
0/4 turns
Rolling-chat turn checks
| Person · turn | Filled input | First word | Recent · pinned · old | Result | Run |
|---|---|---|---|---|---|
1 · 1 | null | not recorded s | null · null · unassessed | Failed | |
1 · 2 | null | not recorded s | null · null · unassessed | Failed | |
1 · 3 | null | not recorded s | null · null · unassessed | Failed | |
1 · 4 | null | not recorded s | null · null · unassessed | Failed |
Method
Staggered multi-turn fictional archive, exact engine-tokenized rolling gateway; independent recent, pinned and old probes each turn.
Limits
Bounded tuning pilot; incomplete window growth cannot pass the full workload grade. Lab retrieval and rolling-history evidence only. No summaries. Shared engine cache intervals include overlapping users and are not per-user hits. Gateway path, not authenticated document ingestion or customer acceptance.
What did not work
- 0/4 requests passed.
What ran
Exact identity
- Original publisher model
- Qwen3.8-27B
- Publisher revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0- Variant · format · precision
- MIRAI-S-TRELLIS-Q8-MTP variant · gguf · MIRAI-S-TRELLIS-Q8-MTP
- Weights SHA-256
af3e2d508cb12ebf68688a0756da6bcb4cdfceca3883f5751666d5e4fb34a334- Engine · version · feature
- Mirai S CUDA ·
0.2.0-dev-vector-disabled· plain - Hardware
- NVIDIA GeForce RTX 3060, 12 GB VRAM
- Harness / workload
keep-rolling-context-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- People in this run
- 1
- Configured window (tokens)
- 262144
- Configured window (tokens)
- 64000
- Output token ceiling
- 256
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
requests ok | 0.00 count | machine total | single | not recorded | requests_ok | |
rolled off abstained | 0.00 count | machine total | single | not recorded | rolled_off_abstained | |
rolled off invalid | 0.00 count | machine total | single | not recorded | rolled_off_invalid | |
rolled off wrong | 0.00 count | machine total | single | not recorded | rolled_off_wrong | |
rolled off correct | 0.00 count | machine total | single | not recorded | rolled_off_correct | |
rolled off total | 0.00 count | machine total | single | not recorded | rolled_off_total | |
requests total | 4.00 count | machine total | single | not recorded | requests_total |
Data and citation
Skilak Keep evidence, run 94cf38e463f04b01a0e25758, measured 2026-10-09T15:29:57.086713+00:00. https://skilak.ai/keep/resources/evidence/r/94cf38e463f04b01a0e25758
