SkilakKeep
Request a pilot

Report · resources

Answer quality

Qwen3.6-35B-A3B · NVFP4 variant · NVIDIA NGC vLLM 0.29

Measured results / Answer quality / Run dae6c36fbef44bb583b17235

Measured 2026-10-07 04:19 UTC · Record date 2026-10-07

Failed. 0/0 tasks

Runs on this setup

Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.

History and measured sweep
History of this setup, newest first
DateShapeEach person · medianMachine totalFirst word · medianResultRun
2026-10-07
1 people
not recorded
not recorded
0.06 s
Partial
2026-10-07
1 people
not recorded
not recorded
not recorded
Failed

Answer checks

0/0 tasks

Method

Deterministic API suite core-v1; SHA256 42c4dc34a066f791a62c9b1dbeb97943988c746309a43a464f0417a02ae9f1da; no model judge; no retries; unchanged endpoint cache; latency excludes local grading; paired calls counted separately. Client concurrency; configured sequence limit 20.

Limits

Bounded fictional tasks only. Unsupported cases are separately counted, never passes. Original publisher name with variant; exact variant repository and revision retained privately. Not customer acceptance.

What did not work

  • Bounded fictional tasks only.

What ran

Exact identity

Original publisher model
Qwen3.6-35B-A3B
Publisher revision
not recorded
Variant · format · precision
NVFP4 variant · safetensors · NVFP4
Weights SHA-256
not recorded
Engine · version · feature
NVIDIA NGC vLLM · 0.29.0+5013de39.dev · plain
Hardware
NVIDIA GB10, 128 GB unified memory
Harness / workload
keep-api-suite-run-v1 · 1
Workload suite · version
core-v1 · 1
Harness SHA-256
not recorded
Fixture SHA-256
42c4dc34a066f791a62c9b1dbeb97943988c746309a43a464f0417a02ae9f1da

Workload shape

People in this run
1
Configured window (tokens)
262144
KV cache precision
fp8
Feature
plain

Full reproduction pack available to customers and evaluators on request

All recorded numbers

Metric definitions, scope and statistics
All aggregate numeric result fields in this run
MetricValueScopeStatisticSample countBasis / source fieldRun
skipped cases
0.00 count
machine total
single
not recorded
skipped_cases
correctness failures
0.00 count
machine total
single
not recorded
correctness_failures
workload total
0.00 count
machine total
single
not recorded
workload_total
workload passed
0.00 count
machine total
single
not recorded
workload_passed
completeness failures
0.00 count
machine total
single
not recorded
completeness_failures

Data and citation

Download this run · OLIR JSONAll numbers · CSVOLIR schema
Skilak Keep evidence, run dae6c36fbef44bb583b17235, measured 2026-10-07T04:19:53.741073+00:00.
https://skilak.ai/keep/resources/evidence/r/dae6c36fbef44bb583b17235

Ready to scope it?

Start with one workload.

Tell us who uses it, what it can read, and what must stay inside. We will map the build.

Request a pilot