Report · resources

Structured tasks

Qwen3.8-27B · INT4 variant · NVIDIA NGC vLLM 0.29

Measured results / Structured tasks / Run 33a18763535def254c3e5a1f

Measured 2026-10-05 03:27 UTC · Record date 2026-10-05

Passed. 16/16 tasks

Runs on this setup

Earlier runs keep their own engine versions and outcomes. Workload shapes stay separate.

History and measured sweep

History of this setup, newest first
Date	Shape	Each person · median	Machine total	First word · median	Result	Run
2026-10-05	1 people	not recorded	not recorded	0.81 s	Passed	Run c8b7c076 →
2026-10-05	1 people	not recorded	not recorded	8.60 s	Partial	Run f4c10f9c →
2026-10-05	unknown people	not recorded	not recorded	not recorded	Passed	Run 33a18763 →
2026-10-05	1 people	not recorded	not recorded	not recorded	Not graded	Run 7abd98af →
2026-10-04	1 people	not recorded	not recorded	not recorded	Not graded	Run 3671f7ed →

Answer checks

16/16 tasks

Method

Synthetic structured-context tasks; no retries, no warmup; unchanged endpoint cache; all planned tasks remain in denominator.

Limits

Lab measurement; broader quality and customer acceptance remain unassessed.

What ran

Exact identity

Original publisher model: Qwen3.8-27B
Publisher revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Variant · format · precision: INT4 variant · safetensors · INT4
Weights SHA-256: not recorded
Engine · version · feature: NVIDIA NGC vLLM · 0.29.0+5013de39.dev · plain
Hardware: NVIDIA GB10, 128 GB unified memory
Harness / workload: not recorded
Workload suite · version: not recorded
Harness SHA-256: not recorded
Fixture SHA-256: not recorded

Workload shape

Configured window (tokens): 8192
Feature: plain

Full reproduction pack available to customers and evaluators on request

All recorded numbers

Metric definitions, scope and statistics

All aggregate numeric result fields in this run
Metric	Value	Scope	Statistic	Sample count	Basis / source field	Run
workload total	16.00 count	machine total	single	not recorded	workload_total	Run 33a18763 →
workload passed	16.00 count	machine total	single	not recorded	workload_passed	Run 33a18763 →

Data and citation

Download this run · OLIR JSON All numbers · CSV OLIR schema

Skilak Keep evidence, run 33a18763535def254c3e5a1f, measured 2026-10-05T03:27:00.672910+00:00.
https://skilak.ai/keep/resources/evidence/r/33a18763535def254c3e5a1f

Ready to scope it?

Start with one workload.

Tell us who uses it, what it can read, and what must stay inside. We will map the build.

Request a pilot