- query p50 secondsquery_p50_seconds · s
- 17.52 sper person · median · sample count not recorded
- query p95 secondsquery_p95_seconds · s
- 73.60 sper person · p95 · sample count not recorded
Answer checks
8/40 correct and cited
Method
Versioned fictional documents; authenticated Open WebUI upload, retrieval and chat with deterministic answer/citation grading. Recall and MRR at ten use labelled source passages; failed questions remain in the total denominator. A fresh system nonce prevents shared-prefix latency reuse.
Limits
Lab cohort, not customer qualification. Reranking and application BM25 are separate recipe arms. Qdrant storage excludes model/document storage. Exact LLM and service pins are retained in the private source record.
What ran
Exact identity
- Original publisher model
- all-MiniLM-L6-v2
- Publisher revision
1110a243fdf4706b3f48f1d95db1a4f5529b4d41- Variant · format · precision
- float32 variant · safetensors · float32
- Weights SHA-256
- not recorded
- Engine · version · feature
- SentenceTransformers inside Open WebUI ·
5.5.1· plain - Hardware
- NVIDIA GB10, 128 GB unified memory
- Harness / workload
keep-rag-trial-v1 · 1- Workload suite · version
not recorded- Harness SHA-256
not recorded- Fixture SHA-256
not recorded
Workload shape
- answer model
- Qwen/Qwen3.6-35B-A3B
- answer variant
- nvfp4; fp8 KV; with multi-token prediction
- corpus sha256
- 51e025e244e9860a3a9a91d69f910bc75ddcbc33c717d4ca8e957501a5a15ae2
- dimensions
- 384
- dtype
- float32
- query generation
- on
- reranker model
- not recorded
- Thinking
- off
Full reproduction pack available to customers and evaluators on request
All recorded numbers
Metric definitions, scope and statistics
| Metric | Value | Scope | Statistic | Sample count | Basis / source field | Run |
|---|---|---|---|---|---|---|
ingest seconds | 85.65 s | machine total | single | not recorded | ingest_seconds | |
citation correct | 8.00 count | machine total | single | not recorded | citation_correct | |
query p50 seconds | 17.52 s | per person | median | not recorded | query_p50_seconds | |
recall at 5 | 0.31 ratio | machine total | single | not recorded | recall_at_5 | |
retrieval p50 seconds | 0.05 s | per person | median | not recorded | retrieval_p50_seconds | |
retrieval p95 seconds | 0.07 s | per person | p95 | not recorded | retrieval_p95_seconds | |
qdrant bytes | 798,007,135.00 bytes | machine total | single | not recorded | qdrant_bytes | |
query p95 seconds | 73.60 s | per person | p95 | not recorded | query_p95_seconds | |
mrr | 0.20 ratio | machine total | single | not recorded | mrr | |
answer and citation correct | 8.00 count | machine total | single | not recorded | answer_and_citation_correct | |
completed questions | 39.00 count | machine total | single | not recorded | completed_questions | |
answer correct | 8.00 count | machine total | single | not recorded | answer_correct | |
questions | 40.00 count | machine total | single | not recorded | questions | |
indexed chunks | 833.00 count | machine total | single | not recorded | indexed_chunks | |
pages | 800.00 count | machine total | single | not recorded | pages | |
recall at 10 | 0.63 ratio | machine total | single | not recorded | recall_at_10 | |
retrieval questions | 32.00 count | per person | single | not recorded | retrieval_questions |
Data and citation
Skilak Keep evidence, run 0934d05eaa71498ab5692df9, measured 2026-10-08T16:54:01.409699+00:00. https://skilak.ai/keep/resources/evidence/r/0934d05eaa71498ab5692df9
