SkilakKeep
Request a pilot

Report · resources

Many people. Full context.

Our specialty: long inputs, many people at once. Measured on real hardware, with a run behind every result.

Updated · 93 setups · 284 runsJSONCSVSchemaChanges

Keep resources / Measured results

NVIDIA GeForce RTX 3060, 12 GB VRAM. Memory figures are whole-host unless marked.

People at once × filled context

Room for the whole team.

KTS-1 runs in progress. Published grids will appear here as standard test suites complete.

Method and terms · Export grid →

Headline results

One job, one measured run. Text jobs use the current default where one is published.

KTS-1 runs in progress. Published headline results will appear here as standard test suites complete. Historical runs remain in the archive below.

Historical archive

Historical pre-standard observations and earlier exploratory sweeps on this hardware, with original measurements and setup history. Each run page contains its workload shape, request rows and provenance links.

Charts and answer checks

KTS-1 runs in progress. Comparison charts and answer checks will appear here as standard test suites complete.

Chat and code9 setups · 3 passed
Chat and code, current setups
SetupWorkload and settingsMeasured resultsResultRun
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 36.06 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 12.52 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
workload total: 8 countmachine total · single · sample count not recorded
workload passed: 8 countmachine total · single · sample count not recorded
Passed
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0
workload total: 16 countmachine total · single · sample count not recorded
workload passed: 16 countmachine total · single · sample count not recorded
Passed
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 2.88 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 13.04 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 6.05 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 12.82 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 17.76 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 12.8 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 2.71 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 12.85 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0 · thinking: false
chat decode tps: 33.08 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 12.58 GiBmachine total · max · sample count not recorded
Not graded
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: Q8_0
chat decode tps: 37.35 tok/sper person · mean · sample count not recorded
code ttft mean s: 0.3 sper person · mean · sample count not recorded
code decode tps: 41.48 tok/sper person · mean · sample count not recorded
Whole-host unavailable peak: 13.39 GiBmachine total · max · sample count not recorded
Passed
Long documents2 setups · 2 passed
Long documents, current setups
SetupWorkload and settingsMeasured resultsResultRun
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
1 people · distinct · cold · 250,000 filled content tokens
First word: 1270.43 / 1270.43 sper-person median / maximum
3/3 Hidden facts found
1/1 completed · 1 overlapping decodes
300 s reference: missed
12.85 GiB whole-host peak
Software thermal slowdown: unassessed
Passed
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
1 people · distinct · cold · 250,000 filled content tokens
First word: 1282.11 / 1282.11 sper-person median / maximum
3/3 Hidden facts found
1/1 completed · 1 overlapping decodes
300 s reference: missed
12.87 GiB whole-host peak
Software thermal slowdown: unassessed
Passed
Rolling chat2 setups · 0 passed
Rolling chat, current setups
SetupWorkload and settingsMeasured resultsResultRun
Qwen3.8-27B · MIRAI-S-TRELLIS-Q8-MTP variant · Mirai S CUDA dev
concurrency: 1 · configured window: 64000
roll ttft max seconds: 92.4 sper person · max · sample count not recorded
requests ok: 4 countmachine total · single · sample count not recorded
roll ttft p50 seconds: 92.4 sper person · median · sample count not recorded
rolled off invalid: 0 countmachine total · single · sample count not recorded
rolled off wrong: 0 countmachine total · single · sample count not recorded
normal ttft max seconds: 62.26 sper person · max · sample count not recorded
rolled off correct: 1 countmachine total · single · sample count not recorded
normal ttft p50 seconds: 49.26 sper person · median · sample count not recorded
rolled off total: 1 countmachine total · single · sample count not recorded
requests total: 4 countmachine total · single · sample count not recorded
Failed
Qwen3.8-27B · MIRAI-S-TRELLIS-Q8-MTP variant · Mirai S CUDA dev
concurrency: 1 · configured window: 64000
requests ok: 0 countmachine total · single · sample count not recorded
rolled off invalid: 0 countmachine total · single · sample count not recorded
rolled off wrong: 0 countmachine total · single · sample count not recorded
rolled off correct: 0 countmachine total · single · sample count not recorded
rolled off total: 0 countmachine total · single · sample count not recorded
requests total: 4 countmachine total · single · sample count not recorded
Failed
Answer quality1 setups · 0 passed
Answer quality, current setups
SetupWorkload and settingsMeasured resultsResultRun
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
concurrency: 1 · configured window: 262144 · kv dtype: q8_0
skipped cases: 8 countmachine total · single · sample count not recorded
correctness failures: 2 countmachine total · single · sample count not recorded
api cases passed: 117 countmachine total · single · sample count not recorded
api cases total: 120 countmachine total · single · sample count not recorded
completeness failures: 1 countmachine total · single · sample count not recorded
Partial

Method and terms

How to read and compare these runs

Writing speed (decode, tok/s) measures output after the first word. Concurrent streams may pause during other people’s prefill. First word after includes scheduling and reading the prompt.

Each person describes one request’s experience; median and mean are labelled. Machine total (aggregate) measures the whole cohort over a stated window. Per-person rates are never summed to estimate it.

New (cold) means no reusable prompt KV state. Already read (warm) means the prefix was processed earlier in the same server lifetime. Warm results never supply reading speed; cached-token counts stay unknown when the engine did not record them.

The grid isolates each model, variant, engine version and generic feature. Cold columns use distinct documents; warm cells label shared inputs. Omitted shapes remain in the historical archive. A cell links to one run, never a blend of setups. The 300-second first-word reference is independent of exact-answer and completion checks. A linked note sends runs with missed checks to the bottom ledger.

Configured window is the server allowance. Filled input is the measured prompt, including the chat template where token usage is available. Completion, exact answer checks, decode overlap and the first-word reference are independent gates.

These are finite lab observations. They establish neither general answer quality nor customer delivery qualification. The workload and public method are listed on each run.

Data and changes

Download the published dataset or use Open Local Inference Result (OLIR) for your own runs. One record describes one run; every numeric metric carries scope, statistic, unit and its original field.

Full dataset · JSONOne row per number · CSVAll runs · OLIR JSONOLIR schemaOLIR formatChangelog · JSONDataset SHA-256
Publication history

Cite a single result using the citation on its run page. Original measurement dates are preserved; republishing does not refresh a measurement.

What did not work

  • Rolling chat: 3 earlier runs missed checks. Full run details remain available.
  • Answer quality: 1 runs missed one or more API or correctness checks.
  • KTS-1 standard: 1 gate or block runs did not pass their check.
Full list · 5 failed or partial runs
Failed and partial runs and startup attempts, newest first
DateResultSetupTriedWhat happenedRun
2026-10-10
Failed
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP variant · Mirai S CUDA dev
KTS-1 standard
Standard check failed; the retained internal record contains the failure reason.
2026-10-09
Failed
Qwen3.8-27B · MIRAI-S-TRELLIS-Q8-MTP variant · Mirai S CUDA dev
Rolling chat
Bounded tuning pilot; incomplete window growth cannot pass the full workload grade.
2026-10-09
Failed
Qwen3.8-27B · MIRAI-S-TRELLIS-Q8-MTP variant · Mirai S CUDA dev
Rolling chat
Bounded tuning pilot; incomplete window growth cannot pass the full workload grade.
2026-10-09
Failed
Qwen3.8-27B · MIRAI-S-TRELLIS-Q8-MTP variant · Mirai S CUDA dev
Rolling chat
0/4 requests passed.
2026-10-09
Partial
Qwen3.8-27B · Mirai S trellis with Q8_0 MTP GGUF variant · Mirai S CUDA dev · with multi-token prediction
Answer quality
119/122 tasks passed. 117/120 API cases passed.

Ready to scope it?

Start with one workload.

Tell us who uses it, what it can read, and what must stay inside. We will map the build.

Request a pilot