SkilakKeep
Request a pilot

Report · resources

Many people. Full context.

Our specialty: long inputs, many people at once. Measured on real hardware, with a run behind every result.

Updated · 93 setups · 284 runsJSONCSVSchemaChanges

Keep resources / Measured results

NVIDIA A10, 24 GB. Memory figures are whole-host unless marked.

People at once × filled context

Room for the whole team.

KTS-1 runs in progress. Published grids will appear here as standard test suites complete.

Method and terms · Export grid →

Headline results

One job, one measured run. Text jobs use the current default where one is published.

KTS-1 runs in progress. Published headline results will appear here as standard test suites complete. Historical runs remain in the archive below.

Historical archive

Historical pre-standard observations and earlier exploratory sweeps on this hardware, with original measurements and setup history. Each run page contains its workload shape, request rows and provenance links.

Charts and answer checks

KTS-1 runs in progress. Comparison charts and answer checks will appear here as standard test suites complete.

Chat and code2 setups · 2 passed
Chat and code, current setups
SetupWorkload and settingsMeasured resultsResultRun
Qwen2.5-7B · vLLM
concurrency: 1 · configured window: 8192
single user decode tps: 30.7 tok/sper person · median · sample count not recorded
Passed
Qwen2.5-72B-Instruct-AWQ · AWQ variant · vLLM
concurrency: 1 · configured window: 8192
single user decode tps: 43.8 tok/sper person · median · sample count not recorded
Passed

Method and terms

How to read and compare these runs

Writing speed (decode, tok/s) measures output after the first word. Concurrent streams may pause during other people’s prefill. First word after includes scheduling and reading the prompt.

Each person describes one request’s experience; median and mean are labelled. Machine total (aggregate) measures the whole cohort over a stated window. Per-person rates are never summed to estimate it.

New (cold) means no reusable prompt KV state. Already read (warm) means the prefix was processed earlier in the same server lifetime. Warm results never supply reading speed; cached-token counts stay unknown when the engine did not record them.

The grid isolates each model, variant, engine version and generic feature. Cold columns use distinct documents; warm cells label shared inputs. Omitted shapes remain in the historical archive. A cell links to one run, never a blend of setups. The 300-second first-word reference is independent of exact-answer and completion checks. A linked note sends runs with missed checks to the bottom ledger.

Configured window is the server allowance. Filled input is the measured prompt, including the chat template where token usage is available. Completion, exact answer checks, decode overlap and the first-word reference are independent gates.

These are finite lab observations. They establish neither general answer quality nor customer delivery qualification. The workload and public method are listed on each run.

Data and changes

Download the published dataset or use Open Local Inference Result (OLIR) for your own runs. One record describes one run; every numeric metric carries scope, statistic, unit and its original field.

Full dataset · JSONOne row per number · CSVAll runs · OLIR JSONOLIR schemaOLIR formatChangelog · JSONDataset SHA-256
Publication history

Cite a single result using the citation on its run page. Original measurement dates are preserved; republishing does not refresh a measurement.

What did not work

No failed or partial runs in this hardware view.

Ready to scope it?

Start with one workload.

Tell us who uses it, what it can read, and what must stay inside. We will map the build.

Request a pilot