- First word afterttft_median_s · s
- 0.06 sper person · median · sample count not recorded
Runs on this setup
Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.
History and measured sweep
| Date | Shape | Each person · median | Machine total | First word · median | Result | Run |
|---|---|---|---|---|---|---|
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Failed | |
2026-10-19 | 1 people | not recorded | not recorded | 0.06 s | Partial | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Partial | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Failed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-19 | 64 people · shared · warm | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 32 people · shared · warm | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 20 people · shared · warm | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 16 people · shared · warm | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 8 people · shared · warm | not recorded | not recorded | not recorded | Capacity limit — confirmed | |
2026-10-19 | 64 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 32 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 20 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 16 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 8 people · distinct · cold | not recorded | not recorded | not recorded | Capacity limit — confirmed | |
2026-10-19 | 64 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 32 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 20 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 16 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-19 | 8 people · distinct · cold | not recorded | not recorded | not recorded | Capacity limit — confirmed | |
2026-10-18 | 64 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 32 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 20 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 16 people · distinct · cold | not recorded | not recorded | not recorded | Capacity limit — confirmed | |
2026-10-18 | 64 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 32 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 20 people · distinct · cold | not recorded | not recorded | not recorded | Above confirmed limit — not attempted | |
2026-10-18 | 16 people · distinct · cold | not recorded | not recorded | not recorded | Capacity limit — confirmed | |
2026-10-18 | unknown people | not recorded | not recorded | not recorded | Passed | |
2026-10-18 | unknown people | not recorded | not recorded | not recorded | Passed |
Answer checks
111/114 tasks109/112 API cases
| Category | Passed / total | Skipped | Run |
|---|---|---|---|
api | 7/8 | 0 | |
code | 16/16 | 0 | |
format | 12/12 | 0 | |
multi_turn | 8/8 | 0 | |
multilingual | 6/6 | 0 | |
numeric | 16/16 | 0 | |
reasoning | 0/0 | 4 | |
refusal | 6/6 | 0 | |
structured_extraction | 15/16 | 0 | |
supplied_context | 15/16 | 0 | |
tools | 0/0 | 8 | |
untrusted_text | 8/8 | 0 | |
vision | 0/0 | 4 |
Method
Deterministic API suite core-v2; SHA256 0ad0b13a94ad74bcdefe99d12d796b175f56e5777d9642a1dd4c0f170d708398; no model judge; no retries; unchanged endpoint cache; latency excludes local grading; paired calls counted separately. Client concurrency; configured sequence limit 20.
Limits
Bounded fictional tasks only. Unsupported cases are separately counted, never passes. Original publisher name with variant; exact variant repository and revision retained privately. Not customer acceptance.
What did not work
- 111/114 tasks passed.
- 109/112 API cases passed.
What ran
Exact identity
- Original publisher model
- Qwen3.6-35B-A3B
- Publisher revision
not recorded- Variant · format · precision
- NVFP4 variant · safetensors · NVFP4
- Weights SHA-256
- not recorded
- Engine · version · feature
- NVIDIA NGC vLLM ·
0.29.0+5013de39.dev· with multi-token prediction - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-api-suite-run-v1 · 1- Workload suite · version
core-v2 · 2- Harness SHA-256
not recorded- Fixture SHA-256
0ad0b13a94ad74bcdefe99d12d796b175f56e5777d9642a1dd4c0f170d708398
Workload shape
- People in this run
- 1
- Configured window (tokens)
- 262144
- KV cache precision
- fp8
- Feature
- with multi-token prediction
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
skipped cases | 16.00 count | machine total | single | not recorded | skipped_cases | |
correctness failures | 2.00 count | machine total | single | not recorded | correctness_failures | |
ttft median s | 0.06 s | per person | median | not recorded | ttft_median_s | |
workload total | 114.00 count | machine total | single | not recorded | workload_total | |
workload passed | 111.00 count | machine total | single | not recorded | workload_passed | |
api cases passed | 109.00 count | machine total | single | not recorded | api_cases_passed | |
ttft p95 s | 0.09 s | per person | p95 | not recorded | ttft_p95_s | |
completion p95 s | 0.52 s | per person | p95 | not recorded | completion_p95_s | |
api cases total | 112.00 count | machine total | single | not recorded | api_cases_total | |
completion median s | 0.19 s | per person | median | not recorded | completion_median_s | |
completeness failures | 1.00 count | machine total | single | not recorded | completeness_failures |
Data and citation
Skilak Keep evidence, run 525c0716346b4c9b81526ec3, measured 2026-10-19T08:01:15.141355+00:00. https://skilak.ai/keep/resources/evidence/r/525c0716346b4c9b81526ec3
