- query p50 secondsquery_p50_seconds · s
- 17.76 sper person · median · sample count not recorded
- query p95 secondsquery_p95_seconds · s
- 40.65 sper person · p95 · sample count not recorded
Answer checks
7/40 correct and cited
Method
Versioned fictional documents; authenticated Open WebUI upload, retrieval and chat with deterministic answer/citation grading. Recall and MRR at ten use labelled source passages; failed questions remain in the total denominator. A fresh system nonce prevents shared-prefix latency reuse.
Limits
Lab cohort, not customer qualification. Reranking and application BM25 are separate recipe arms. Qdrant storage excludes model/document storage. Exact LLM and service pins are retained in the private source record.
What ran
Exact identity
- Original publisher model
- all-MiniLM-L6-v2
- Publisher revision
1110a243fdf4706b3f48f1d95db1a4f5529b4d41- Variant · format · precision
- float32 variant · safetensors · float32
- Weights SHA-256
- not recorded
- Engine · version · feature
- SentenceTransformers inside Open WebUI ·
5.5.1· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rag-trial-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- answer model
- Qwen/Qwen3.6-35B-A3B
- answer variant
- nvfp4; fp8 KV; with multi-token prediction
- corpus sha256
- 51e025e244e9860a3a9a91d69f910bc75ddcbc33c717d4ca8e957501a5a15ae2
- dimensions
- 384
- dtype
- float32
- reranker model
- not recorded
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
ingest seconds | 85.12 s | machine total | single | not recorded | ingest_seconds | |
citation correct | 7.00 count | machine total | single | not recorded | citation_correct | |
query p50 seconds | 17.76 s | per person | median | not recorded | query_p50_seconds | |
recall at 5 | 0.31 ratio | machine total | single | not recorded | recall_at_5 | |
retrieval p50 seconds | 0.05 s | per person | median | not recorded | retrieval_p50_seconds | |
retrieval p95 seconds | 0.06 s | per person | p95 | not recorded | retrieval_p95_seconds | |
qdrant bytes | 798,007,164.00 bytes | machine total | single | not recorded | qdrant_bytes | |
query p95 seconds | 40.65 s | per person | p95 | not recorded | query_p95_seconds | |
mrr | 0.20 ratio | machine total | single | not recorded | mrr | |
answer and citation correct | 7.00 count | machine total | single | not recorded | answer_and_citation_correct | |
completed questions | 40.00 count | machine total | single | not recorded | completed_questions | |
answer correct | 7.00 count | machine total | single | not recorded | answer_correct | |
questions | 40.00 count | machine total | single | not recorded | questions | |
indexed chunks | 833.00 count | machine total | single | not recorded | indexed_chunks | |
pages | 800.00 count | machine total | single | not recorded | pages | |
recall at 10 | 0.63 ratio | machine total | single | not recorded | recall_at_10 | |
retrieval questions | 32.00 count | per person | single | not recorded | retrieval_questions |
Data and citation
Skilak Keep evidence, run 45b9fcd99abd4ef2ac4998c5, measured 2026-10-08T07:06:16.538845+00:00. https://skilak.ai/keep/resources/evidence/r/45b9fcd99abd4ef2ac4998c5
