One file. Zero dependencies. Ready before your editor finishes loading.
A local RAG engine in a single static binary — MiniLM embeddings compiled
inside, hybrid dense + BM25 retrieval, MCP-native. No Node, no Python,
no model download on first query.
Single-binary local RAG MCP server in Rust. MiniLM compiled in, hybrid search, zero deps.
2026-11-08
Stars
—
Tools
—
Uptime
—
Why quillrag
~20 ms to ready
MCP handshake completes before the model even loads
Zero runtime deps
no Node, no Python, no pip/npm, no model downloads — ever
Hybrid retrieval
dense cosine ⊕ BM25 fused with Reciprocal Rank Fusion
Private by construction
no network code path after installation
One file, three OSes
~105 MB (the model lives inside), CI-built for linux/macOS/Windows
Quick start
# 1. grab a prebuilt binary (or cargo install --path .)
gh release download --repo Ayush-yadav11/quillrag -p '*linux*'
tar xzf quillrag-x86_64-linux.tar.gz && chmod +x quillrag
# 2. point it at any folder of notes/docs/code
./quillrag index ~/notes # incremental walk
# 3. ask it something
./quillrag search "how does backpropagation work"
Or wire it straight into Claude Desktop / Cursor and let the AI search your
notes mid-conversation — config below.
$ ./quillrag serve --data-dir ~/.local/share/quillrag
2026-08-26 INFO quillrag 0.1.2 ready in 41ms <- handshake-ready before the model loads
Why it's fast
Stage
Cost
Binary start + MCP initialize
~20 ms (measured: store open + tool registration only)
First rag_search / rag_index call
+~300 ms one-time (mmap safetensors, build BERT graph)
Subsequent searches
~25 ms per query (2-core CPU, small corpus)
Re-indexing unchanged corpus
near-zero (FNV content hash skip)
The embedding model is lazy: the MCP handshake and rag_status never touch
it, so editors see an instant server.
Install
Download a prebuilt archive from the latest release
— Windows x86_64, macOS Apple Silicon, and Linux x86_64 are all built by CI on
every version tag:
Linux requirement: v0.1.6 binaries are built on Ubuntu 24.04 runners and
link glibc 2.39 (pidfd_spawnp/pidfd_getpid). On older distros (glibc
< 2.39, e.g. Ubuntu 22.04), build from source with your local toolchain
instead: cargo install --path ..
Or build from source:
cargo install --path .
Cross-compile targets used by CI: x86_64-unknown-linux-gnu,
aarch64-apple-darwin, x86_64-pc-windows-msvc.
Or just run ./quillrag serve and point any stdio client at it.
Tools
Tool
What it does
rag_index
Incrementally index a directory/file. Skips unchanged files, re-embeds only diffs. Pruning is scoped to the directory you pass: docs indexed from other roots are untouched. Symlinked files/dirs are followed.
rag_search
Hybrid retrieval: dense MiniLM cosine + BM25 keyword, fused with Reciprocal Rank Fusion. Returns ranked chunks with source paths.
quillrag index ~/notes # incremental walk
quillrag search "auth flow" -k 5 # one-shot search
quillrag status # stats
quillrag clear # wipe
Design
Embeddings: candle (pure Rust)
running sentence-transformers/all-MiniLM-L6-v2 — masked mean pooling +
L2 norm, numerically matching sentence-transformers on CPU. Weights are
include_bytes!-ed into the binary and mmap'd from a materialized cache on
first load.
Storage: single redb file — chunk text,
raw f32 vectors, document metadata. Atomic commits; crash-safe.
Keywords: tantivy BM25 sidecar
index rebuilt per indexing pass (cheap at pocket scale).
Fusion: Reciprocal Rank Fusion (Σ 1/(60+rank)) — no score-scale tuning,
robust to heterogeneous rankings.
Chunking: paragraph-first with 1000-char cap and 120-char overlap;
oversized paragraphs hard-split at sentence boundaries.
File types indexed by default
md markdown txt rst json yaml yml toml csv tsv html htm xml log rs py js jsx ts tsx go c h cpp hpp java rb sh bash zsh sql proto graphql dockerfile makefile ini cfg conf env — extend with -e ext1,ext2 / "extensions": [...].
Ignored dirs: every dot-directory (.git .obsidian .vscode …) plus
node_modules target dist build venv __pycache__ vendor.
Optional formats (library features)
Applications embedding the engine (default-features = false) can index more
through IndexHooks extractors:
Feature
Adds
pdf
PdfExtractor: text-layer PDFs, one section per page; hits carry page.
ocr
Ocr, ImageExtractor (png/jpg/webp/bmp/gif) and OCR of scanned PDF pages via PdfExtractor::with_ocr. Bundles the ocrs models (~12 MB, CC BY-SA 4.0, see assets/ocr/NOTICE.md); English / basic Latin text only.
Files that are readable but contain no text are recorded as empty documents,
so unchanged ones are not re-OCR'd on every pass (IndexOptions::retry_empty
re-checks them).
Indexing behavior
quillrag index ~/notes # incremental walk of one root
quillrag index ~/notes --no-prune # add/update without deleting anything
quillrag index ~/notes --force # re-embed this root, preserve other roots
Pruning is scoped to the index root. Docs indexed from ~/work-docs
survive a later quillrag index ~/personal-notes against the same data
dir. Only files that lived under the directory being walked and have since
disappeared get pruned. Existing files excluded by extensions or size stay
indexed. --no-prune disables deletion, including through MCP's no_prune.
--force re-embeds only the selected sources. Use clear explicitly to wipe
a store. An empty successful scan warns and may prune that root's last file.
Symlinks are followed. Staging trees built from symlinked source dirs
index normally. Cycles, broken links, and unreadable directories abort the
walk before changing the store. Paths retain symlink aliases, so indexing
both an alias and its target creates separate entries.
Progress is logged per 128-chunk embedding batch (stderr), so a long
CPU indexing run is visibly alive.
Upgrading existing indexes
Older batched indexing could associate document paths with the wrong chunks.
The fix cannot reconstruct those associations from the stored index alone.
Build a fresh data directory from your source files, verify search results,
then switch your client to it. Keep the old directory until verification passes.
This also removes legacy relative-path keys, which the new indexer preserves
rather than guessing their original working directory. New keys are absolute.
Paths containing .. resolve through the filesystem; use a stable spelling
for symlink-based roots. Re-indexing refreshes the BM25 sidecar as well.
Privacy & footprint
Everything runs locally: embeddings, storage, search. Nothing leaves the
machine — there is no network code path at all after installation.
Binary ≈ 105 MB (the model lives inside). RAM ≈ 120 MB resident while idle,
spiking to ~250 MB during batch embedding.
Scaling & limits
quillrag stores everything in a single redb file and runs dense retrieval as
an exact, single-threaded linear scan over all vectors — no ANN index yet.
That makes the relevant limit query latency, not storage. Storage scales to
millions of chunks; retrieval speed is O(N) per query.
Corpus
Vectors
Approx. RAM (f32)
Steady-state query
1K chunks
1K
~1.5 MB
~25 ms (measured)
10K chunks
10K
~15 MB
~250 ms (extrapolated)
100K chunks
100K
~154 MB
~2–5 s (extrapolated)
1M chunks
1M
~1.5 GB
20–60 s (extrapolated — not viable without ANN)
Verified on a corpus of 1K chunks (5/5 tests including real JSON-RPC-over-stdio
e2e); figures above 1K are extrapolated from the O(N) dense-scan cost, not
measured. A synthetic scale probe (src/bin/quillbench.rs) exists to measure
the curve on your own hardware — run cargo build --release && ./target/release/quillbench.
What this means in practice:
Great fit: personal/local knowledge bases, project docs, notes, code —
up to low-tens-of-thousands of chunks where sub-second-to-interactive latency
holds.
Away from the sweet spot: corpora in the hundreds of thousands+ where you
need interactive (<200 ms) retrieval — you'll want an ANN index (see Roadmap).
How it compares to common alternatives on the relevance axis:
Embedding-only (e.g. raw FAISS flat / simple vector store): same
all-MiniLM-L6-v2 ceiling as quillrag's dense path, but quillrag adds BM25 +
RRF fusion, which wins on keyword-heavy queries (error codes, IDs, exact
tokens). quillrag has no reranker or metadata filtering, which llama-index
offers on top.
llama-index local backends: functionally similar hybrid retrieval
(BM25 + vector + RRF). quillrag trades llama-index's rich reranking/parent-child
chunking/query-expansion for a zero-dependency single binary and instant
startup. Relevance on a standard dataset (BEIR/MS MARCO) is not yet
benchmarked — see the open issue tracking ANN + a relevance baseline.
Roadmap
quillrag is deliberately minimal today. The big unlock is an approximate
nearest-neighbor index:
ANN (HNSW / IVF) over the dense vectors — turns O(N) scan into
sub-millisecond ANN lookup, pushing the interactive ceiling from ~10K to
millions of chunks on a single machine.
Quantization (PQ / SQ) — drops vector RAM from 4 bytes/dim to ~1 byte/dim,
so 1M chunks ≈ 380 MB instead of 1.5 GB.
Multi-threaded scan — parallelize the current exact path as a stopgap.
Reranker hook — optional cross-encoder rerank of the fused top-k.
Relevance benchmark — BEIR / MS MARCO nDCG@10 vs. llama-index baselines.
Track the ANN work here: issue #1 — "ANN index for <1M chunks."
FAQ
Is it really one file? Yes. The MiniLM weights + tokenizer are compiled in
via include_bytes!. No npm install, no Python, no model download on first
query. The binary is ~105 MB because the model lives inside it.
Why is startup so fast? The embedding model is lazy. The MCP handshake and
rag_status never touch it — editors see a ready server in ~20 ms. The model
only loads on the first rag_search / rag_index (~300 ms one-time).
What's the largest corpus it handles? Verified at 1K chunks (~25 ms/query).
The architecture scales to millions of stored chunks; interactive retrieval
holds up to low-tens-of-thousands today, and an ANN index (Roadmap) extends that
to 1M+.
How is this different from llama-index? Similar hybrid retrieval quality, but
quillrag is a single static binary with no runtime/dependency footprint and
instant startup. llama-index adds rerankers, sophisticated chunking, and query
expansion that quillrag doesn't have yet.
What file types are indexed?md markdown txt rst json yaml yml toml csv tsv html htm xml log rs py js jsx ts tsx go c h cpp hpp java rb sh bash zsh sql proto graphql dockerfile makefile ini cfg conf env — extend with -e.
Does it phone home? No. There is no network code path after installation.
Changelog
v0.1.6: root-scoped pruning, working --no-prune, source-scoped
--force, symlink traversal with failure-safe discovery, absolute source
keys, and correct partial-batch progress. Fixes shuffled document/chunk
associations in batched writes and missing BM25 result paths. Rebuild old
indexes into a fresh directory as described above.
v0.1.5: withdrawn draft. Initial safety fixes were incomplete; do not
use this tag for production.
v0.1.3 — MCP tool descriptions rewritten for clarity, parameter semantics,
and behavioral transparency (read-only/destructive flags, usage guidance);
server.json shipped in-repo for MCP Registry publishing.
v0.1.2 — skip all dot-directories when indexing (.obsidian plugin configs
no longer pollute results); first fully automated 3-platform CI release.
Upgrade note: run quillrag clear once and re-index.
v0.1.1 — CI-built release artifacts for linux/macos/windows with checksums.
v0.1.0 — initial public release; renamed from pocketrag.
Development
cargo test --release # unit + end-to-end with real model inference
cargo run -- serve # dev server
RUST_LOG=debug cargo run ... # verbose logs (stderr only)
License: MIT. The optional OCR models under assets/ocr/ are CC BY-SA 4.0;
see assets/ocr/NOTICE.md.