Memory Industry

MemoryIndustry

Formerly cuba-memorys. Same daemon, same cuba_* MCP tools, new product name.

Long-term memory for AI coding agents. An MCP server that gives your agent a knowledge graph it can search, reason over, and be corrected by — so it stops forgetting your codebase between sessions.

Written in Rust. Backed by PostgreSQL + pgvector. 31 MCP tools (32 with CUBA_DOCS=1), 28 CLI commands, and every number below measured on a benchmark that — as of v0.12 — actually measures what it claims to. (The previous one did not. See Measured.)

Install

pip install memory-industry        # or: npm install -g memory-industry
claude mcp add memory-industry -- memory-industry

# Previous names still install the same binary:
#   pip install cuba-memorys
#   npm install -g cuba-memorys

That is the whole setup. On first run it provisions a PostgreSQL 18 + pgvector container via Docker and initializes the schema. Docker must be running. The cuba-memorys command remains a binary alias.

{
  "mcpServers": {
    "memory-industry": {
      "command": "memory-industry"
    }
  }
}

No DATABASE_URL needed. Or run cuba-memorys setup (or memory-industry setup) and it writes the config for every client it finds — then cuba-memorys setup check audits them for disagreement, which is the failure that actually bites (two configs, two embedding dimensions, one silently broken search).

{
  "mcpServers": {
    "memory-industry": {
      "command": "memory-industry",
      "env": { "DATABASE_URL": "postgresql://user:pass@localhost:5432/brain" }
    }
  }
}

Needs the vector and pg_trgm extensions. cuba-memorys doctor will tell you if anything is missing.

stdio gives every client its own process, and every process loads its own copy of the models — embeddings, reranker and NLI together are several GB. Three editor windows meant three copies, and on a 16 GB laptop that is the whole machine.

serve loads them once and answers every client over loopback HTTP, which is also the shape the 2026-07-28 MCP specification settled on: no session handshake, every request self-describing.

cuba-memorys serve                      # 127.0.0.1:8787 by default
cuba-memorys serve 127.0.0.1:9000       # or pick the address

memory-industry serve is the same command. Point every client at it, and give each one its own Mcp-Client-Id so their sessions stay separate — without it jornada start in one window becomes the active session of the next:

{
  "mcpServers": {
    "memory-industry": {
      "type": "http",
      "url": "http://127.0.0.1:8787/mcp",
      "headers": { "Mcp-Client-Id": "editor-window-1" }
    }
  }
}

GET /health reports uptime, database reachability and how many clients have been seen. status is three-valued — ok, starting, degraded — and the code is always 200: a 503 would take out of rotation a daemon that still serves lexical search and still serves the one endpoint that can explain what is wrong. Alert on status, not on the code.

GET /identity and POST /pair are how a client finds out that the daemon it reached is the right one and pairs with it. GET /identity answers {service, host_id, name} to anyone and, given a credential id and a nonce, a proof only a host that holds that credential's hash can compute (or revoked / unknown, with no proof). POST /pair redeems a pairing code and answers a MAC only a host that knows the code can compute. Both refuse a request that carries a forwarding header or a foreign Origin, /pair takes at most 4 KiB, answers every failure with the same 401 {"error":"pairing failed"}, and counts its failures in the same per-address brake as /mcp (10 per minute).

With CUBA_HTTP_TOKEN in an Authorization: Bearer header the answer also carries runtime: mode, resource_tier, {state, device, reason} for the embedder, reranker and NLI — device is where the session actually opened (gpu, or cpu with the provider's cause in reason when the GPU provider refused to start) and null for a model that has no session yet or never opened one — gpu{build, degraded, placement}, where placement is only what the plan asked for and degraded also turns true when a provider refused, and llm{configured, backend, model, base_url} — which is how you find out where the reranker is actually running without reading the source. Without the token none of that is served, and neither is graph_db.last_error, which can name an internal host and port. No key, no DATABASE_URL and no model path is ever printed, with or without a token.

The token is mandatory if you bind anything other than loopback — the daemon serves the entire graph with no authentication by default — and on a routable address it must be at least 32 characters.

The port opens first, then the models load, and only then does the daemon announce itself and start serving. Binding first keeps the cheap diagnosis — a second daemon on the same port fails immediately instead of spending two minutes loading before it finds out — and connections that arrive during the load queue in the kernel backlog, so a client waits and gets a real answer rather than a refused connection. If the load overruns MEMORY_INDUSTRY_WARM_BEFORE_SERVE_SECS (180 s) the daemon serves anyway, /health says starting, and every search asking for reranking comes back marked degraded. Under stdio a client that gives up at 30 s without closing stdin used to leave an abandoned multi-GB process holding every model it had loaded; stdio now exits if no handshake arrives within MEMORY_INDUSTRY_HANDSHAKE_TIMEOUT_SECS (60 s, 0 or off disables).

That queue only helps a client that can outwait the load: one whose own budget is shorter — a sync peer gives up after CUBA_HANDLER_TIMEOUT_SECS (30 s) — still cuts the call, and cuts it seeing a timeout instead of a refused connection, which is the better diagnosis but is not the real answer the queue was there to buy. /health is what says which of the two states the daemon is in, and it answers two different questions. That it answers at all means the port is served, which is everything a caller needs when what it is about to call does not touch the models — a sync fetch moves rows. That it answers ready: true means the models finished loading as well, which is what to wait for before a search that asks for reranking, or when the first search must not be the one that pays for the load. The first wait is bounded and the second is not: the daemon opens within MEMORY_INDUSTRY_WARM_BEFORE_SERVE_SECS whatever the models are doing, but a warm-up that overruns that ceiling keeps serving everything with ready: false for as long as it takes, so ready: true can fail to arrive on a daemon that is answering perfectly. Wait on it without a ceiling of your own and you hang on a healthy machine.

Without a model, embeddings are hash-based: deterministic, and semantically meaningless. Search still works through the lexical and BM25 branches, but nothing understands meaning.

One command installs the models and the ONNX runtime, on any OS — no shell scripts, no manual ORT_DYLIB_PATH:

cuba-memorys models all          # embeddings + NLI + reranker + runtime
cuba-memorys models embed        # just the embeddings model (~113 MB)
cuba-memorys models all --gpu    # GPU runtime, if you have one
cuba-memorys doctor              # confirms what loaded

Everything lands in ~/.cache/memory-industry/ (or in ~/.cache/cuba-memorys/ on an install from before the rename) and is found automatically. models downloads only when you run it — nothing is fetched behind your back. A download goes to <dir>.downloading first and becomes <dir> only once every file is down and verified, so a failed one leaves no empty directory for the loaders to pick.

Moving an old cache under the new name. The ~/.cache/cuba-memorys/ root is still read for this release, but each file decides between the two roots on its own, and one entry created under the new name — pgpass_app at startup, undo/ from delete — is enough for models to download again what the old root already holds. doctor warns while the old root has anything in it. One command moves it:

memory-industry cache migrate            # the plan: touches nothing
memory-industry cache migrate --apply    # moves it

An entry only in the old root moves; one identical in both loses its old copy. One that differs keeps the copy the binary reads today — for pgpass_app the one next to the pgpass in use, for a model directory the one that actually holds the model — and the other is set aside as <name>.legacy-YYYYMMDD in the new root, never deleted. A file another process holds (on Windows, the runtime a running daemon loaded) is skipped and the command exits non-zero: stop the daemon and run it again. What already moved stays moved, and a second run says there is nothing to migrate.

bge-m3 (1024-d) is better than e5-small for Spanish, though the size of the gap is no longer claimed (the old +21 nDCG figure came from a broken benchmark). It needs a dimension migration (scripts/migrate-embedding-dim.sh 1024) and CUBA_EMBED_MODEL=bge-m3 CUBA_POOLING=cls.

CUBA_MODE is a preset that sets the database, the models, and outbound network together, so you pick one name instead of lining up a dozen env vars:

Two machines, one memory. Point both at the same managed Postgres (Neon or Supabase free tier both have pgvector and fit the 36 MB corpus many times over), give each a name with CUBA_NODE_NAME, and CUBA_MODE=red. What one writes, the other reads; every memory records which machine it came from (origin_node). Without a shared database, cuba_sync does the same job through a git repository — see Sync between machines. Do not expose your own Postgres port to the internet — use a managed provider's TLS, or a private network like Tailscale.

Real isolation when you share. A shared database is where row-level security stops being decorative. Run cuba-memorys secure once (as the admin role) to create a non-superuser cuba_app with RLS and append-only audit actually enforced, then point the runtime at it with CUBA_SKIP_MIGRATIONS=1. cuba-memorys doctor reports whether the runtime role is a superuser (which bypasses all of it) or not.

Maximum capability. CUBA_MODE=completo turns on the cross-encoder reranker (+93% nDCG) and cuba_docs. The reranker no longer needs that mode when the machine can actually run it: a build with a GPU provider that finds a working device turns it on by itself, because that is where it fits its budget. On CPU it stays off by default — the table below is why — and cuba-memorys doctor says which of the three reasons applies. Asking for rerank: true in the call still overrides everything. On CPU faro time-boxes it and falls back to the RRF ranking (CUBA_RERANK_TIMEOUT_SECS, default 20 s), so a slow machine still answers. GPU binaries ship with CUDA (NVIDIA) and, on Windows, DirectML (any GPU) — cuba-memorys models runtime --gpu fetches the accelerated runtime.

Fetching the GPU runtime is only half of it: the binary itself has to be built with --features cuda, or gpu::configure() registers no provider and the reranker runs on CPU. That is not a hypothetical — it is what a 50-candidate rerank costs on a 6-core laptop, measured with cargo run --release --example rerank_bench:

Same ranking either way — CPU and GPU agree candidate for candidate, differing only in the fifth decimal of the score. Run rerank_bench on any machine to see whether the reranker fits its budget there or is silently throwing the work away, and cuba-memorys doctor reports whether this build has a GPU provider at all.

This section used to say "every model quietly runs on CPU", implying all three would run on the GPU once you built with --features cuda. Only the reranker ever did. The embedder ships dynamically quantised to INT8, which means 96 DynamicQuantizeLinear feeding 144 MatMulInteger — and the CUDA provider registers no kernel for either, so ONNX Runtime partitions them onto the CPU no matter what you build. Registering CUDA for that session bought nothing and cost a VRAM arena the model never computed in: 374 MiB held while all 544 MB of weights sat in host RAM. The NLI cross-encoder has the opposite problem — it is FP32 and stuck there, because mDeBERTa is documented upstream as not supporting FP16 and the INT8 build returns confident false entailments.

So placement is now decided per model rather than once per process, and only the reranker asks for the GPU. On the 6 GB card this was measured on, the daemon went from 5228 MiB of VRAM to 2950 MiB while searching, and 0 while idle — and down to 1460 MiB with the two opt-in steps in Footprint below.

Individual env vars (CUBA_DOCS, CUBA_RERANKER_PATH, …) always override the preset.

What it actually does

Most memory servers are a key-value store with an embedding bolted on. This one models four kinds of memory, because the psychology literature says they are four different things and they decay differently:

Procedural memory is a separate table rather than a ninth observation type for a specific reason: ACT-R separates declarative memory (reinforced by access) from procedural (reinforced by success). As an observation, a recipe consulted constantly because it keeps failing would climb in importance. It is ranked by Wilson lower bound, so 1/1 successes scores 0.21 and 47/50 scores 0.84 — a lucky first try does not outrank a track record.

Retrieval

Hybrid RRF fusion (k=60, Cormack 2009) over three signals — full-text, BM25 (ts_rank_cd), and pgvector HNSW — with entropy-routed weighting that shifts from keyword-heavy to semantic as the query's Shannon entropy rises.

Answers arrive in compact by default: abbreviated keys, content truncated at 1200 chars. 30% fewer tokens, and a slightly better nDCG — measured on the 221 id-scored questions, +0.0090 with a paired 95% interval of [+0.0024, +0.0166]. The format genuinely cannot change which documents rank; what it changes is how many of them survive the response token budget before they are scored. Verbose at the default 5000-token budget weighs 5286 tokens and loses its tail; compact weighs 3723 and keeps it. Pass "format": "verbose" for the full per-branch score breakdown.

Verification that actually verifies

cuba_faro mode=verify checks a claim against what is stored. It used to score claims by cosine similarity to the retrieved evidence, and that does not work — similarity measures what a text is about, not what it asserts. "cuba-memorys is written in Rust" and "…in Java" are nearly the same vector. Measured on the live corpus, the false claim scored 0.61 and the true one 0.59.

Entailment is a different question from similarity, and it needs something that reads. A local cross-encoder now judges each piece of evidence — supports / contradicts / unrelated — and confidence is derived from the verdicts, each weighted by that evidence's similarity. Same corpus, after:

Being on-topic is not support, and unrelated counts for neither side.

The judge is mDeBERTa-v3-base-xnli running locally on ONNX: 100 languages, ~50 ms per verdict, no API key, no network, no cost. That matters here — about 75% of this corpus is Spanish, and the English-only NLI models everyone reaches for first would have silently failed on three memories out of four. Install it with cuba-memorys models nli; cuba-memorys doctor will tell you whether it loaded.

Without it, verification falls back to an LLM (your MCP client's own model via sampling, a local claude CLI, or the Anthropic API) — and with none of those, to an honest unknown rather than an invented verdict.

Two things it will not do. It will not confirm a claim on weak evidence: entailment must clear 0.80 while contradiction needs only 0.60, because confirming a false memory and doubting a true one are not errors of equal cost. And when it cannot tell, it says so instead of returning whichever number came out largest — an argmax over a 3-way head will happily publish supports for a claim that is flatly false, and did.

Calibrated abstention

The out-of-distribution gate rejects queries the corpus cannot answer. The threshold is not a magic constant: Ledoit-Wolf covariance shrinkage plus a conformal quantile, calibrated against your own corpus with cuba-memorys calibrate --dataset <questions.jsonl> --apply and persisted (the dataset is required — without it the command refuses). (The theoretical χ² threshold rejected 100% of answerable queries. Distribution-free calibration is not a nicety here.)

Sync between machines, through git

CUBA_MODE=red puts two machines on one database. cuba_sync is the other route, for machines that never see each other: the graph is written out as JSON you can commit, and read back on the other side.

cuba-memorys sync export            # write the bundle under .memory-industry/ (or $CUBA_SYNC_DIR)
cuba-memorys sync import            # read one back in
cuba-memorys sync diff              # entities on disk vs entities in the database
cuba-memorys sync status            # which bundles this machine has already imported
cuba-memorys hook install           # export after every commit, import after every checkout, merge and rebase

The same four actions are cuba_sync action=export|import|diff|status. A bundle is one JSON file per entity with its observations inside, plus episodes/YYYY-MM/, errors/, decisions/, relations.json, projects.json, tombstones.json and a manifest.json — the active project and anything not bound to a project, unless you pass --scope all. Embeddings stay out unless you ask for them (--with-embeddings): they are most of the bytes and they can be recomputed. A bundle imports once, and the manifest hash covers the contents of every file in it — so an unchanged bundle is skipped, and a hand-edited entity file is a new bundle rather than a silent no-op. An export writes only the files whose contents changed: exporting a database nobody wrote to since the last export touches nothing, manifest.json included, whose exported_at is therefore the time the bundle last changed.

hook install imports after a merge and a rebase, too. It writes four hooks: post-commit exports, post-checkout imports, post-merge imports, and post-rewrite imports after a rebase. git runs neither of the first two for a git merge, and no post-merge for a git rebase (what git pull --rebase runs on a branch with commits of its own), so without the last two a merged or rebased bundle never reached the database and the next commit exported the database over it, dropping the other side's rows. githooks(5) says post-rewrite "is invoked by commands that rewrite commits (git-commit when called with --amend and git-rebase)", with amend or rebase as its first argument; after an amend it does nothing, since the amended commit is this machine's own export. A merge git stops on runs no post-merge; the commit that concludes it has a second parent, and on that commit post-commit imports the resolution before it exports. Installed before this? Run hook install again.

hook install writes down the directory it resolved. Every hook sets CUBA_SYNC_DIR to the directory install resolved — a relative value is read against the repository root, not the directory you ran it from — because a commit made from an IDE, a GUI client or a cron job brings its own environment, and without the fixed value that commit exported somewhere the merge driver was not looking. Change the variable, run hook install again: it replaces its own block instead of keeping the first one. The .gitattributes line is written the only way git matches one, relative to the top of the work tree with / and no . or .. (quoted when the name has a space), and it replaces any line an earlier install left; before, ./dir, an absolute path or data\sync on Windows were written as given, matched nothing, and were reported as added. A CUBA_SYNC_DIR outside the repository still gets the hooks — they export to and import from the shared folder — but no merge driver and no line, because git never merges files outside its work tree; install says so: .gitattributes: skipped — <path> is outside this repo; git never merges it.

The merge driver merges what it knows and leaves the rest to you. Entities, relations, projects, episodes, errors and decisions are recognised by where they sit under the sync directory (entities/<file>, relations.json, episodes/<YYYY-MM>/<file>, …), whatever that directory is called, and merged by id. tombstones.json is merged as one set keyed by (table_name, row_id): both branches' deletions, once each, and where both deleted the same row the later deletion stands. facts.json, procedures.json, artifacts.json and source_trust.json are merged by the key the import conflicts on — fact_id, id, (path, project_id), source — keeping, where both sides hold a row, the later one: a closed fact over an open one, the later updated_at of a procedure (its success and failure counts at the greater of the two), the later CRDT clock of an artifact, and each count of a source's trust at its highest. manifest.json merges as yours when both sides agree on project_id, with_embeddings, embedding_dim, embedding_model and schema_version — what the import acts on without checking it against the files; the rest is derived and the next sync export rewrites it. When one of them differs it is a conflict, and the message names the field. embeddings.bin.zst, any other file, and any file whose two sides do not parse whole — or carry a field this build does not know — make it exit non-zero, so git leaves a conflict. It used to exit 0 without writing, which git reads as "merged": your side was kept and the other one dropped with no marker.

A deletion travels now, and stops where it would take something with it. Deleting a row records a tombstone, and the receiving side deletes exactly the ids that were named. Before this, a delete was not slow to arrive — it was undone: the peer still had the row, exported it, and it came back on the next round trip. The entity tombstone is the dangerous one, because deleting an entity cascades to everything hanging off it. It is applied only when this machine has no observations or episodes under that entity that the sender never named; otherwise it is withheld and reported in tombstones_withheld. A tombstone for an entity with three children there must not take three hundred here.

And a bundle cannot quietly wipe you. If the tombstones in it would delete at least 25 rows and more than 10% of the observations on this machine, the import refuses and asks for confirm=true. A remote wipe and a large legitimate cleanup look identical; the only difference is whether you meant it. The floor matters as much as the ratio: on a database with a single observation a pure percentage demanded confirmation to delete that one, and a guard that trips on ordinary curation is one everybody learns to pass confirm=true through — and then it guards nothing.

conflict=merge does not merge content, and now says so. merge and skip are one policy: rows that are missing here arrive, and where a row already exists with different content, the one that was here first wins and the incoming text is dropped. What changed is the silence — the import counts those rows and reports them as diverged, with their ids and a note saying what it did. conflict=overwrite takes the incoming version and keeps the one it replaced in previous_versions (the newest 20 are kept), and clears the embedding when the content changed, so a row stops being retrievable by a meaning it no longer carries.

Counters do merge, under either policy. importance and access_count on an entity, and strength on a relation, are not values one side copies from the other: each machine grows its own, from its own reinforcement and its own traversals. The higher of the two wins, which is idempotent — importing the same bundle twice inflates nothing. (Summing would be more faithful to "both machines counted", and would double on a re-import, so it loses to a rule that cannot corrupt the number.)

Which machine is which. Each installation generates a uuid in its own database on first migration — one row, stable across restarts, unique by construction — and the manifest carries it, so a bundle can say which machine produced it. CUBA_NODE_NAME keeps meaning what it always meant: a human-readable label stored in origin_node. It is not the identity and could not be one, because two machines both called pop-os is the likeliest outcome there is.

The clock ticks for what a peer needs, and stays still for local noise. An observation's version advances when its content, type, trust, evidence level or tags actually change, and for nothing else. Decay moves importance and last_accessed; reembed replaces vectors. If either woke the clock, every export would ship a graph that had not changed and the two machines would never stop talking to each other about nothing. Rewriting a row with the same content does not tick it either, so an idempotent re-import does not invent a conflict out of agreement.

Older bundles still import. The format is SCHEMA_VERSION 2: version, updated_at, origin_node, previous_versions, evidence, verification and trust travel now, because a conflict rule that compares clocks needs the clock to be in the file. Bundles written before that still import — every new field defaults, and a v1 observation lands as asserted, which is the honest reading of a file that never claimed anything stronger.

Anything in an incoming bundle that looks like a credential is stored quarantined instead of trusted — withheld from cuba_faro and cuba_expediente until you promote it with cuba_eco — because an import reads JSON out of a repository anyone with push access can write to.

A peer that only ever reads. CUBA_PEER_TOKEN reaches five more verbs and nothing else. pull returns the bundle in the response instead of writing it anywhere, paged by file (limit, offset) up to a 3 MB budget per page — abort if manifest_hash changes between pages, because that means this node was written to mid-transfer and the pages describe two different states. notify is the one write a peer token may make: a short summary (at most 2000 characters) saying the other machine learned something, tagged with node_id/node_name, surfaces at the next cuba_jornada start and in status, and closes itself when a bundle carrying its manifest_hash is imported — it never enters the graph itself. conflicts lists the rows two machines disagree about with both texts, and resolve id=… keep=ours|theirs|both closes one: keep=both (the default) keeps this machine's text current and files the other in previous_versions, discarding nothing, while theirs also clears the embedding because it described text that is no longer here. fetch is the other half and runs on the local machine: it pages a peer's pull over HTTP, lands the files, imports them with the same validation as any bundle, and records the peer's manifest hash so the next fetch stops before opening a transaction when nothing changed. Embeddings are omitted by default on export and included by default on pull — a peer that receives text without vectors cannot search what it just received until it re-embeds, which on a machine without a GPU is slow and sequential — and a bundle whose model or dimension does not match this machine is refused rather than silently filling the index with vectors from another space.

And it tells you when it is broken

$ cuba-memorys doctor
[  ok  ] migrations           49 aplicadas, ninguna dirty
[  ok  ] embedding_dim        runtime 1024-d == columna vector(1024)
[  ok  ] runtime_role         'cuba_app' sin superuser — RLS y audit efectivos
[ warn ] binary_freshness     4 proceso(s) MCP corren un binario más viejo que el de disco

This exists because the failure mode of a hybrid search engine is not a crash — it is a vector branch dying and the search quietly becoming lexical, with no symptom. The server now refuses to start on an embedding-dimension mismatch, and search sets degraded: true in the response when a branch fails.

The CLI: your memory without an LLM in the middle

Twenty-three commands. memory-industry --help lists them all.

dedupe — because a different string is a different entity

cuba_alma create inserts with ON CONFLICT (name). So one project fragments into Mapupita-Web, Mapupitta-Web (typo), Mapupita Web, mapupita… and searching one finds none of the others. On a real 266-entity graph, 158 of them (59%) had not a single relation — for PageRank and multi-hop retrieval, they did not exist.

What decides a merge is not the embedding centroid. That was the obvious idea and it is wrong: M-Codes Reference Guide and G-Codes Reference Guide sit at 0.811 cosine between centroids. On a corpus about one domain, centroid similarity measures the domain, not the entity — a 0.80 threshold would have merged two different CNC guides, irreversibly.

So --apply merges only what is provable (identical after normalizing case and separators). Typos and near-matches are shown, and judged one at a time with --judge. The old name is written to brain_entity_aliases, so nothing is lost: looking it up still resolves.

The 31 tools

Named after Cuban culture. cuba-memorys advertises all of them, or set CUBA_TOOL_PROFILE=lean to advertise an everyday core of 13 plus cuba_tools + cuba_call — 15 of 31, a 49% smaller catalogue with zero functions lost, the rest reachable on demand.

Knowledge graph — cuba_alma (entities) · cuba_cronica (observations, episodes, timeline) · cuba_puente (typed relations, traversal, link prediction) · cuba_ingesta (bulk import)

Search — cuba_faro (hybrid RRF, verification, MMR diversification, OOD abstention)

Error memory — cuba_alarma (report) · cuba_remedio (resolve) · cuba_expediente (search past errors; warns if an approach failed before)

Sessions & decisions — cuba_jornada (session lifecycle, diff) · cuba_decreto (architecture decisions) · cuba_proyecto (per-project isolation) · cuba_pre_compact (survive /compact)

Procedural — cuba_receta (recipes ranked by Wilson lower bound)

Cognition — cuba_reflexion (gap detection) · cuba_hipotesis (abductive inference) · cuba_contradiccion (semantic conflicts) · cuba_juez (LLM judge) · cuba_centinela (prospective triggers) · cuba_calibrar (Bayesian calibration, source credibility)

Maintenance — cuba_zafra (decay, prune, merge, PageRank, Leiden communities) · cuba_eco (RLHF feedback) · cuba_vigia (health, drift, centrality) · cuba_forget (GDPR erasure) · cuba_archivo (CFR-21 hash-chain audit log) · cuba_pizarra (working memory) · cuba_sync (git-friendly export/import between machines, with propagated deletions and a remote-wipe guard)

Meta — cuba_tools (discover) · cuba_call (invoke)

Configuration

Footprint

A memory server is infrastructure: it is running when you are not using it. On the 6 GB laptop GPU this was measured on, it used to hold 5228 MiB of VRAM from boot — 93% of the card — and other GPU programs stopped being able to start. The NVIDIA driver was returning NV_ERR_NO_MEMORY on channel creation, which is what a game or a GPU-accelerated terminal fails on.

Two of the numbers below ship as defaults; two need a line of config, and this table keeps them apart rather than quoting the best one as if it came free.

Everything in the defaults column is code that ships. CUBA_RERANK_CHUNK=4 is one env var. The last column additionally needs the rebuilt reranker described below. None of it removed a feature.

Four things got it there:

Placement per model, not per process. Only the reranker is accelerated by a GPU — the INT8 embedder cannot be, and the FP32 NLI is not worth a gigabyte of VRAM for a judge that runs occasionally and tolerates 150-400 ms. The arena cap is per session, so three sessions asking for CUDA on a 6 GB card is a 3× overcommit waiting to fail.

A CUDA arena that stops doubling. ArenaExtendStrategy::NextPowerOfTwo is the ONNX Runtime default and it reserves in powers of two rather than what the session asked for.

The reranker loads on its first batch. Under socket activation the daemon starts far more often than it reranks, and plenty of those starts only ever answer a save.

A daemon that is not running when nobody is asking. CUBA_IDLE_SHUTDOWN_SECS plus a systemd .socket unit: the socket owns the port, the daemon starts on the first real connection and exits after the idle window. It shuts down through the normal path — serve returns, the background drain flushes in-flight embedding writes, sqlx closes its pool — because exiting the process directly loses those writes silently.

# ~/.config/systemd/user/memory-industry.socket
[Socket]
ListenStream=127.0.0.1:8787
Accept=no

[Install]
WantedBy=default.target

# ~/.config/systemd/user/memory-industry.service (abridged)
[Service]
Type=exec
ExecStart=%h/.local/bin/memory-industry serve 127.0.0.1:8787
# Every knob — CUBA_IDLE_SHUTDOWN_SECS, the per-model devices — lives in the env file, never here.
EnvironmentFile=%h/.config/memory-industry/memory-industry.env
# An idle shutdown exits 0 — Restart=always would bounce it straight back up.
Restart=on-failure

Both units, and the env file that offers every key, are generated by the binary and versioned in packaging/: memory-industry setup service prints the plan and --apply installs them. ExecStart has to name the binary you actually installed — command -v memory-industry — and a wrong path fails as status=203/EXEC. The cuba-memorys.service / cuba-memorys.socket pair earlier releases shipped left the repository in 0.28.0; if you still have them installed, see the 0.28.0 entry in the CHANGELOG.

serve adopts the socket systemd passes as fd 3 (LISTEN_FDS), so the port is held while the daemon is not running and no client sees a refused connection.

The unit must also bind loopback. With socket activation the .socket unit's ListenStream decides the address and CUBA_HTTP_ADDR is ignored, so serve checks the address of the socket it is handed and refuses a routable one unless CUBA_HTTP_TOKEN is set.

Host RAM: it sizes itself to your machine

VRAM was only half of it. The weights also live in host memory, and that appetite used to be fixed no matter what the machine had. Measured with cargo run --release --features cuda --example mem_bench, daemon stopped, on the 6 GB laptop GPU:

Resident settles near 1941 MiB; the peak is 2677 because loading a 1,1 GB ONNX file costs transient memory on top of the weights it leaves behind. The peak is the number that has to fit, not the steady state.

On the machine this was measured on that is fine. On a 4 GB laptop it is not, and under a systemd unit capped at MemoryHigh=4500M it has been seen paging 2,56 GiB to swap — MemoryHigh does not kill, it reclaims, and reclaiming is paging.

Two traps worth knowing if you re-run this. mem_bench attributes VRAM to its own PID via nvidia-smi --query-compute-apps, because reading memory.used charges you for every other process on the card — that is how a first attempt showed 3590 MiB "at process start" that belonged to a game and a desktop shell. And run it with the daemon's own environment: with CUBA_RERANKER_PATH unset it silently loads the unfused artifact and the warm-up goes from 3,7 s to 131 s on CPU.

So the daemon now reads the machine at startup and picks a level. Nothing is invented for this: all three degradations already existed and are tested.

How the level is chosen. The budget is min(cgroup limit, system available) − 768 MiB of headroom, and the cgroup has to win. On this machine /proc/meminfo reports 7,16 GB available while the daemon's cgroup caps it at 4,39 GiB — believing /proc would load 2,6 GiB of weights against a limit where the kernel already starts paging. The reader walks from the cgroup root down to the leaf and takes the tightest memory.max or memory.high it finds, because the limit is usually set on an ancestor.

Models are then fitted in order of measured value: the embedder first, then the reranker (+93% nDCG, so it outranks the judge), then NLI.

The plan can only take away. Every knob is capped at the value the daemon already used, so on a machine with room the level is full and nothing changes. Degradation only goes downward.

You always win. Any of these set by hand is left untouched — the regulator fills gaps, it does not overwrite decisions:

CUBA_EMBED_INTRA_THREADS   CUBA_RERANK_INTRA_THREADS   CUBA_NLI_INTRA_THREADS
CUBA_RERANK_CHUNK          CUBA_GPU_MEM_LIMIT_MB       CUBA_OOD_FIT_LIMIT
CUBA_DB_MAX_CONNECTIONS

To force a model off regardless of the budget, point it at a path that does not exist — CUBA_RERANKER_PATH=/nonexistent or CUBA_NLI_PATH=/nonexistent. That is the same mechanism the regulator itself uses.

To see what it decided, run cuba-memorys doctor: it reports the reading and the resulting plan, and warns when the level falls to minimal. The plan is also logged at startup with the full machine reading behind it.

The reranker artifact

The published bge-reranker-v2-m3 ONNX is converted to FP16 before any graph fusion, which leaves 785 Cast nodes threaded through it. ONNX Runtime claws some of that back at load time (2023 → 897 nodes, 49 SkipLayerNormalization), but it cannot fuse Gelu and it repeats the work on every cold start. Rebuilding from the FP32 export and fusing first:

python -m onnxruntime.transformers.optimizer \
  --input model.onnx --output model.onnx \
  --model_type bert --num_heads 16 --hidden_size 1024 \
  --opt_level 1 --use_gpu --float16

Identical top-10 order on a real search, fused_score differing by at most 0,0029; on synthetic logits at the real batch shapes, Pearson ≥ 0,9997 with the same ranking in every batch.

Attention does not fuse, and that is not fixable here. is_fully_optimized: Attention (or MultiHeadAttention) not fused, at opt_level 0, 1, 2 and 99, on both the FP16 artifact and the clean FP32 one. The export builds its Q/K/V reshapes from dynamic shape subgraphs (Shape → Gather → Unsqueeze → Concat → Reshape) and AttentionFusion needs a Reshape with a constant shape to read num_heads and head_size off it. So flash/efficient attention stays unavailable without a re-export using static shapes — worth knowing before anyone spends an afternoon on it.

Measured — and the benchmark that was lying

Until v0.12 this section carried a line reading "every number here is measured rather than assumed", and every number in it was wrong. The benchmark was broken in three ways, and finding out cost two published conclusions.

It had ten queries. A 95% interval of roughly ±0.12; the smallest effect it could detect was ~0.25 nDCG. Any claim about a smaller difference was noise wearing a decimal point.

Relevance was judged by substring match. A result counted as correct if its text merely contained a marker word — so every observation mentioning "postgres" scored as a right answer to any question about postgres, whether it answered anything or not. That measures keyword presence, not retrieval, and it tilts the whole benchmark toward the lexical branch and against the vector one.

nDCG normalized against what was retrieved, not what exists. With 5 relevant documents in the corpus and 2 found, the "ideal" ranking was taken to be those 2 — so a system that missed 60% of the answer scored a perfect 1.0. (And R@10 = 3.125 shipped in this file. Recall is a proportion.)

The real number is not 0.894. On 221 id-scored queries it is nDCG@10 = 0.50 [95% CI 0.44–0.56]. The system did not get worse. It was never 0.894.

What that cost

• "The cross-encoder reranker earns nothing" — it had never run. Three bugs in series: faro wrapped the call in if let Ok(..) and dropped the error; it fed token_type_ids to a model that is XLM-RoBERTa and has none; it read f16 logits as f32. The output was "bit for bit identical" to no reranking not because reranking changed nothing, but because it never happened. Fixed; being measured properly now.

• Associative retrieval does degrade — but the old evidence (−0.03 at n=10) could not have shown it. On the new dataset with a paired bootstrap (the correct test: same queries in both arms), the interval is [−0.051, −0.018] and never touches zero. It improves 0 queries and hurts 23. The decision was right; the reasoning was not. The power was never in more data — it was in using the right test.

What survives, re-measured honestly

Foundations

Development

git clone https://github.com/LeandroPG19/Memorys.git
cd Memorys/rust && cargo build --release

# On an NVIDIA machine, build this way instead — without it the reranker spends
# its whole budget for a ranking that gets discarded. It accelerates the
# reranker only; see Footprint for why the other two models stay on the CPU.
cargo build --release --features docs,cuda

./scripts/demo.sh                  # runs on a throwaway Postgres it removes on exit
./scripts/merge-gate.sh            # local CI 100% — sole merge judge (see docs/gate.md)
./scripts/como-el-ci.sh todo       # same SIL
./scripts/quality-gate.sh          # second judge: CRAP + mutants of rust/src in the diff
cargo run --release --example rerank_bench   # does the reranker fit its budget here?

Publishing is tag-driven: v* on LeandroPG19/Memorys triggers GitHub Release binaries (5 platforms), PyPI wheels (memory-industry and the cuba-memorys alias), npm (memory-industry and the cuba-memorys alias), and the MCP Registry (io.github.LeandroPG19/memory-industry). A test pins Cargo.toml, package.json, pyproject.toml and server.json so those versions cannot drift.

License

Apache-2.0 — use it, modify it, ship it, sell it, embed it in a closed product. No copyleft obligation. The licence also grants patent rights explicitly, which is the part legal departments care about.

Author

Leandro Perez G. — @LeandroPG19