# Open Local Inference Result (OLIR) 0.1 OLIR describes one measured inference run on one machine. Numeric results retain their run ID, unit, scope, statistic, sample count and original field. The JSON Schema is the normative 0.1 contract. Unknown reproduction information is explicit; it is never reconstructed from an intended command or a current engine registry. ## Record and identity `format` is `olir/0.1`. `id` is opaque. `measured_at` preserves the timestamp recorded by the harness (some historic captures recorded completion time). `record_date` preserves the original evidence date. `published` is null when the first publication date was not captured; republishing is not a measurement. `publisher`, `scenario`, `system`, `model`, `engine`, `settings`, `workload`, `metrics`, `requests`, `quality`, `outcome`, `caveats`, `reproduce` and `history` are closed objects. Unknown keys are rejected. The schema defines the approved settings and request vocabulary. Model identity uses its original publisher and a neutral variant. Weights file hashes identify recorded artifacts without publishing private download locations or community revisions. Only verified vendor image digests cross the Keep publication boundary. `settings.context_tokens` retains its legacy meaning: filled content target on long-document runs, configured window on other kinds. When a runtime manifest records it, `settings.configured_context_tokens` is the server window explicitly. `workload.input_tokens` gives min/median/max of tokenizer-reported filled prompts, including the chat template. It stays null for old short-prompt smokes that did not capture usage. Configured context is never proof of tested context. ## Metrics Every metric requires `name`, `source_field`, `value`, `unit`, `scope`, `statistic`, `n` and `basis`. Scope is `per_person` or `machine_total`. Statistics are `median`, `mean`, `p95`, `max` or `single` (one observation, counter, fraction numerator/denominator or instance value). A null `n` means the sample denominator was not recorded. Counts are measurements, not means. Optional numeric `samples` and sample `stddev` are included where recorded. `source_field` preserves the original field and request-array position. This supports legacy adapters without silently relabelling their measurements: - `decode_tps` is writing speed after first output. The team-load `per_request_decode_tps` source is a **mean**. A separately calculated `per_person_decode_median_tps` source is the median of recorded request rates. - `aggregate_decode_tps` is the machine total over the first-output-to-last- completion window. `aggregate_e2e_tps` includes the first-word wait. Neither is the sum of the per-person rates. - `prefill_tps` is effective cold reading speed, including queueing. It is invalid for warm requests or a positive cached-token count. Warm cache-served rates are withdrawn from the public projection; immutable private values stay. - `ttft` fields measure time to first word, retaining the stated mean, median, p95 or maximum. `peak_*` fields retain their memory scope (whole host versus CUDA allocator). Quality counts and retrieval ratios keep their suite context. Cold means no reusable KV state for that prompt; warm means a prefix was processed earlier in the same server lifetime. Request `cached_prompt_tokens` is a nonnegative count when the raw usage reported one, otherwise null. Null does not mean zero. Each request retains filled input, output, timings and checks. ## Outcomes and reproduction `outcome.status` is `pass`, `partial`, `fail`, `not_graded` or `did_not_start`. Gates separate completion, retrieval and first-word latency. A reference latency miss can coexist with an immutable run pass for completion and exact retrieval; the reference does not rewrite the old grade. Failed and partial runs have reasons. Quality suite counts are bounded evidence, never a general accuracy score. `reproduce` includes recorded harness and test-suite names/versions and fixture and harness hashes. Missing versions, OS and driver remain null. Public records show what ran: hardware, original model and variant, engine and version, generic feature, users, filled context, cache state, measured results and correctness. The opaque `setup_id` distinguishes configurations without revealing tuning. The historical `recipe` field contains only a generic feature label: plain, with multi-token prediction, or speculative decoding. Tuned settings, internal recipe names, container pins/digests and launch/client commands are private. Full reproduction pack available to customers and evaluators on request via the site's pilot request page. Private envelopes retain the complete record; public withholding never removes a failed run. The metric vocabulary follows llama-bench's prompt/generation split, vLLM's first-token/percentile vocabulary, per-user versus per-system serving analysis, and separate aggregate and instance-level quality records. OLIR is an export contract, not a certification or compliance claim. ## Exports and changes `keepctl evidence publish` generates `public-olir.json`, `public-olir.schema.json`, `public-evidence.csv` and `public-changelog.json` beside the legacy full public dataset. CSV has one row per numeric result field, including numeric per-request/turn/category fields. It carries the opaque run ID, identity, scenario, metric, unit, scope, statistic, sample denominator and source field. Unknown values are blank. JSON is authoritative for nested identity, settings, samples, checks and reproduction gaps. The changelog seeds verified thermal and inventory corrections and records dataset changes by opaque IDs: new runs, corrected projections and superseded headlines. Original private envelopes are immutable. Both old records and their measurement dates remain available. An unchanged publish adds no change entry. See evidence publishing for the privacy boundary, committed-byte verification and the independent public site checks.