Wet

WET - Web Extended Toolkit MCP Server

Renamed (2026-09-13): repo is now wet — CLI-first (wet command). PyPI package stays wet (PyPI policy blocks new project wet); MCP server is a secondary surface: run wet with no subcommand.

mcp-name: io.github.n24q02m/wet

Renamed: the repo, CLI and Python module are now wet. The PyPI distribution stays wet-mcp (install pip install wet-mcp / uvx wet-mcp). The MCP server remains available: run wet with no subcommand (bare = MCP passthrough); the legacy wet-mcp console alias still works.

Open-source MCP server for AI agents: web search, content extraction, and library docs.

Current release: v3.x. media(action="analyze") was removed in the v2.0.0 BREAKING release. Use imagine-mcp's understand action for vision/audio/video analysis. See docs/migration.md for the upgrade recipe.

Table of contents

• Features
• Status
• Quick install
• Self-host usage
• Configuration
• Documentation
• Tools
• CLI
• Comparison
• Security
• Build from Source
• Deploy to Cloudflare
• Smithery
• Trust Model
• License

Features

• Web Search -- Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache (1 h general / 5 min time-sensitive), standardized citation format, and 200-token snippet cap. Optional cloud search backends (Tavily, Brave, Exa, Kagi, OpenRouter, Firecrawl) as a fallback chain via SEARCH_BACKENDS
• Academic Research -- Search Google Scholar, Semantic Scholar, arXiv, PubMed, CrossRef, BASE
• Library Docs -- Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs
• Content Extract -- 5-strategy escalation chain via n24q02m-web-core ScrapingAgent (basic_http -> tls_spoof -> render backends from BROWSER_BACKENDS (native / browserless / cf-browser-rendering) -> optional key-gated captcha), markitdown bridge for low-tier HTML/MD fallback, smart chunks structured output (clean text + markdown + JSON-LD + code blocks + metadata), batch processing (up to 50 URLs), deep crawling, site mapping
• Local File Conversion -- Convert PDF, DOCX, XLSX, CSV, HTML, EPUB, PPTX to Markdown
• Media -- List + download images / videos / audio files. analyze was removed in v2.0.0 -- use imagine-mcp.understand for vision/audio inference
• Anti-bot -- Stealth strategies bypass Cloudflare, Medium, LinkedIn, Twitter
• Zero Config -- Built-in local reference embedding + reranking through fastretrieval, no API keys needed. Optional cloud models configured per task via the [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml (OpenRouter default; one OPENROUTER_API_KEY serves every cell)
• Sync -- Cross-machine sync of indexed docs via Google Drive (OAuth Device Code, no browser redirect)

Quick install

# Method 1 (default): plugin install via Claude Code
/plugin marketplace add n24q02m/claude-plugins
/plugin install wet@n24q02m-plugins

# Method 2 (CLI): direct uvx invocation
claude mcp add wet -- uvx wet-mcp

# Method 3 (source-built container for HTTP / multi-device / OAuth)
docker build --target http -t wet:local .
docker run -d --name wet-http -p 8084:8080 \
  -v wet-data:/data -e PUBLIC_URL=https://wet.example.com \
  wet:local

# Method 4 (remote): point a client at an HTTP deployment
claude mcp add --transport http wet https://<your-host>/mcp

Install matrix (stdio unless noted; see the Setup page for full steps):

Public OCI image publication is discontinued. Existing historical registry tags remain untouched; new container deployments build from source or use the Cloudflare-managed registry.

The HTTP endpoint speaks Streamable HTTP and is OAuth-gated -- your client is prompted to authenticate in the browser on first connect (no API key to paste). Stand one up via Method 3 or the Deploy to Cloudflare section.

Full setup matrices live at the canonical docs site mcp.n24q02m.com/servers/wet/setup/ and the paste-to-agent snippets at claude-plugins/plugins/wet/setup-with-agent.md (per Spec F single source of truth).

Self-host usage

Two supported ways to run the always-listening HTTP server. In both, the MCP endpoint is http://127.0.0.1:8000/mcp (Streamable HTTP) and the config root is ~/.wet/ (config.toml, docs.db, subs/). Never run the server as a spawned subprocess of a client — register the endpoint in your client instead.

Dev (uv, no-auth loopback)

git clone https://github.com/n24q02m/wet && cd wet
uv run wet config init            # writes ~/.wet/config.toml (default: auth = "no-auth")
uv run wet                        # serves http://127.0.0.1:8000/mcp

no-auth refuses non-loopback binds — the server is localhost-only until you switch to token auth.

Always-on (docker)

# 1. Mint a token and its scrypt hash (hull-core canonical; the hash command
#    never echoes the token itself)
openssl rand -hex 32                 # the token — give it to clients, keep it secret
uv run wet token hash <token>        # paste the output as token_hash

# 2. Instance config from the example
cp docker-config/config.example.toml docker-config/config.toml
#    edit [server] token_hash and the [models.*] cells (see below)

# 3. Data continuity (optional): carry an existing instance over by copying
#    docs.db and subs/ into the wet-home named volume BEFORE first start:
docker volume create wet-wet-home && \
  docker cp ~/.wet/docs.db wet-wet-home:/docs.db && \
  docker cp ~/.wet/subs wet-wet-home:/subs  # adjust: files land as root, chown 999:999

docker compose up -d --build

Compose publishes 127.0.0.1:${WET_PORT:-8000} → container 8000 (loopback only) and mounts docker-config/config.toml read-only at /home/appuser/.wet/config.toml. Persistent data (docs.db, subs/) lives in the wet-home named volume; the download/cache dir in wet-data. Editing the config takes effect on docker compose restart.

Consumers

# CLI (same token, JSON-RPC over HTTP not needed — use the installed CLI):
uvx wet-mcp --help                  # or `wet <tool>` for direct tool calls

# MCP client (Claude Code):
claude mcp add --transport http wet http://127.0.0.1:8000/mcp \
  --header "Authorization: Bearer <token>"

# Raw probe:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \
  -X POST -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'            # 401
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \
  -X POST -H 'Authorization: Bearer <token>' -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}'  # 200

Config: local vs cloud models

The [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml are per-task (each its own base_url + api_key + model, OpenAI-spec). The example ships OpenRouter examples; any OpenAI-compatible endpoint works — cloud, or a local server such as Ollama (base_url = "http://host.docker.internal:11434/v1" from the container). Keys in the file are host-only material; alternatively leave api_key = "" and inject at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY.

Configuration

wet runs zero-config out of the box: web search uses an embedded local SearXNG, and embedding/reranking fall back to the bundled local ONNX models through fastretrieval when no cloud keys are set. For higher-quality results, configure the per-task provider cells in ~/.wet/config.toml. Operational settings are plain environment variables (no app prefix) -- in the HTTP self-host mode they are entered through the browser setup form instead.

Provider cells -- [models.embed|rerank|chat|jev_score] each take their own base_url + api_key + model (OpenAI-spec). OpenRouter is the pre-wired default for every cell; any OpenAI-compatible endpoint works. Cell keys may be left empty in the file and injected at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY. When a cell keeps the OpenRouter default base_url, OPENROUTER_API_KEY in the environment serves it as a last-resort credential — one OpenRouter key can power every task. Empty embed/rerank cells fall back to the local ONNX models; an unkeyed chat cell disables LLM features (e.g. extract(action="agent")).

FASTRETRIEVAL_CACHE_PATH controls the local model cache.

Search backends -- SEARCH_BACKENDS is an ordered runtime fallback chain: searxng (default, local or external via SEARXNG_URL), keyed tavily / brave / exa / kagi / openrouter, optional-key firecrawl, and credential-free duckduckgo / startpage. Keyed providers use TAVILY_API_KEY, BRAVE_API_KEY, EXA_API_KEY, KAGI_API_KEY, or OPENROUTER_API_KEY (comma-separate for rotation). The openrouter backend runs the query through a chat completion with the openrouter:web_search server tool and maps the returned url_citation annotations onto the shared result shape; OPENROUTER_MODEL (default a free-tier model), OPENROUTER_BASE_URL, and OPENROUTER_SEARCH_ENGINE override its behavior. Firecrawl attempts a keyless request when FIRECRAWL_API_KEY is absent; rejection or a DuckDuckGo/Startpage bot challenge advances the chain.

Result reranking uses the configured rerank cell ([models.rerank] — cloud via OpenRouter by default, or local ONNX when unkeyed); the same chain serves web search, research, similar-page search, agent search, and docs discovery/indexing fallbacks. SearXNG retains its science-category filter for research; other providers use their own search capabilities. Hosted users configure the chain and keys in their own relay record. An empty hosted record never inherits an operator's provider key or local SearXNG URL; single-user stdio still uses env/settings and preserves the public local path.

Browser render backends -- BROWSER_BACKENDS (CSV, escalation chain) picks the headless render leg of extract: native (in-process chromium, the zero-config default), browserless (self-host render service -- set BROWSERLESS_URL + BROWSERLESS_TOKEN), and cf-browser-rendering (Cloudflare Browser Rendering -- set CF_ACCOUNT_ID + CF_BROWSER_RENDERING_TOKEN). Empty chain falls back to native. Set CAPSOLVER_API_KEY to append an optional, key-gated CAPTCHA tier as the last escalation step.

Robots policy -- set RESPECT_ROBOTS_TXT=true to enforce robots.txt across both the extract strategy chain and the Crawl4AI-backed crawl, sitemap, and list_media actions. The default is false to preserve existing deployment behaviour; configure this process-level policy explicitly when the operator requires robots enforcement.

Invisible tier + seeded identity (opt-in) -- append invisible to BROWSER_BACKENDS to add a stealth-Firefox engine tier as the last escalation step (n24q02m-hull[invisible] extra: pip install "wet-mcp[invisible]"). One coherent browser identity spans the whole chain: seed it with WET_IDENTITY_SEED (or let wet derive once and persist it under ~/.wet/subs/<sub>/identity.json) and install the wet[identity] extra. Without the extras the chain keeps legacy behaviour (warn-once). STEALTHFOX_BINARY points at a patched Firefox build to skip the engine download; IDENTITY_PROFILE_DIR (default ~/.wet/subs/<sub>/profiles) keeps persistent browser profiles per namespace.

Disable local fallbacks -- opt out of the heavy in-process local fallbacks per capability (e.g. on a slim container that renders/searches/embeds via cloud backends only): DISABLE_LOCAL_BROWSER, DISABLE_LOCAL_SEARCH, DISABLE_LOCAL_EMBED, DISABLE_LOCAL_RERANK.

Docs sync -- SYNC_ENABLED (default true), GOOGLE_DRIVE_CLIENT_ID (required for sync), SYNC_FOLDER (default wet), SYNC_INTERVAL (default 300s). Sync uses Google Drive over the OAuth Device Code flow (no browser redirect). DOCS_DB_BACKEND=cf-d1 disables GDrive/S3 file sync, including automatic startup, relay wizard and device-code setup, even if legacy sync settings remain. Non-CF SQLite deployments retain GDrive sync; SYNC_S3_BUCKET selects S3 instead, and SYNC_ENABLED=false disables both.

HTTP self-host -- MCP_TRANSPORT=http, PUBLIC_URL=<your-domain>. The setup form is gated by MCP_RELAY_PASSWORD; multi-user deployments require CREDENTIAL_SECRET (per-user vault key), MCP_JWT_SIGNING_SECRET (rotatable OAuth JWT key), and MCP_DCR_SERVER_SECRET.

Example stdio config (one OpenRouter key powers every provider cell):

{
  "mcpServers": {
    "wet": {
      "command": "uvx",
      "args": ["wet-mcp"],
      "env": {
        "OPENROUTER_API_KEY": "sk-or-xxx"
      }
    }
  }
}

To use a different endpoint or model per task, edit the [models.*] cells in ~/.wet/config.toml, or inject HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY at start instead of storing keys in the file.

Status

Stable architecture with two transports: stdio (default, local) and HTTP (self-host, OAuth-gated). No daemon-bridge layer and no auto-spawn from stdio. The media.analyze action was removed in the v2.0.0 BREAKING release -- see docs/migration.md for the upgrade recipe. Current release line: v3.x.

Documentation

Full docs at mcp.n24q02m.com/servers/wet/setup/:

• Setup -- install methods for Claude Code, Codex, Gemini CLI, Cursor, Windsurf, mcp.json
• Modes overview -- stdio / local-relay / remote-relay / remote-oauth
• Multi-user setup -- per-JWT-sub credential model

In-repo references (Spec F single source of truth: setup docs live in claude-plugins/plugins/wet/):

• docs/ARCHITECTURE.md -- web-core ScrapingAgent integration, strategy chain, storage layout, LLM provider dispatch
• docs/BENCHMARKS.md -- v1.x baseline coverage / latency placeholders + tier-1 fixture metrics

Install with AI agent -- paste this to your AI coding agent:

Install MCP server wet following the steps at https://raw.githubusercontent.com/n24q02m/claude-plugins/main/plugins/wet/setup-with-agent.md

Tools

6 MCP tools (3 domain + config + help + config__open_relay). The legacy setup tool merged into config action dispatch.

Media boundary: For vision / audio understanding (image captioning, OCR, audio transcription, video summarization), use imagine-mcp. media.analyze was removed in wet v2.0.0 -- use imagine-mcp.understand instead.

CLI

The package installs two console scripts: wet (primary) and wet-mcp (legacy alias kept so existing uvx wet-mcp configs keep working). A bare invocation (or any leading-dash flag) starts the server; a leading positional argument is dispatched as a subcommand.

uvx --from wet-mcp wet warmup   # try a subcommand without a persistent install

wet                             # start the server over stdio (default transport)
wet --http                      # start the server over Streamable HTTP (self-host mode)

wet auth google                 # authorize the Google credential provider for Drive sync
wet logout                      # clear the local Google Drive sync token
wet warmup                      # pre-download local models + run auto-setup (SearXNG, browser) to avoid first-run delays
wet docs reindex <library>      # drop the cached docs index for <library>; the next docs search re-indexes it

auth google accepts an optional bring-your-own OAuth client via --client-id and --client-secret (single-user / local machine only; the token is written to the local store). Each subcommand prints a JSON result and exits.

Security

• SSRF prevention -- URL validation on crawl targets
• Graceful fallbacks -- Cloud → Local embedding, multi-tier crawling
• Error sanitization -- No credentials in error messages
• File conversion sandboxing -- Optional CONVERT_ALLOWED_DIRS restriction

Build from Source

git clone https://github.com/n24q02m/wet.git
cd wet
uv sync
uv run wet

Deploy to Cloudflare

Run your own single-user wet instance serverless on Cloudflare (Containers + D1 + Vectorize + KV).

Prerequisites: a Cloudflare account on the Workers Paid plan — required for Containers, D1, and Vectorize (the Cloudflare free tier does not include them) — and the wrangler CLI.

1. git clone https://github.com/n24q02m/wet && cd wet
2. wrangler login
3. Provision resources and apply the D1 schema:

   wrangler d1 create wet-docs
   wrangler d1 execute wet-docs --file migrations/0001_init_wet.sql --remote
   wrangler d1 execute wet-docs --file migrations/0002_project_context.sql --remote
   wrangler d1 execute wet-docs --file migrations/0003_version_index_state.sql --remote
   wrangler vectorize create wet-docs-vectors --dimensions 768 --metric cosine
   wrangler kv namespace create wet-kv
   

   Paste the returned IDs into wrangler.jsonc.
4. Build the slim HTTP image from this checkout and push it directly to Cloudflare's managed registry (CF Containers cannot pull from external registries):

   docker build --target http --build-arg SLIM=1 -t wet:beta .
   wrangler containers push wet:beta   # prints registry.cloudflare.com/<ACCOUNT_ID>/wet:beta
   

5. Set operator auth/storage and Browser Run secrets:

   wrangler secret put CREDENTIAL_SECRET
   wrangler secret put MCP_JWT_SIGNING_SECRET
   wrangler secret put MCP_RELAY_PASSWORD
   wrangler secret put MCP_DCR_SERVER_SECRET
   wrangler secret put CF_BROWSER_RENDERING_TOKEN
   

6. wrangler deploy and complete setup in the browser relay form at your Worker domain.

Storage maps to Cloudflare via MCP_STORAGE_BACKEND=cf-kv (credentials/tokens, encrypted), DOCS_DB_BACKEND=cf-d1 (docs + BM25 full-text), and Vectorize (embeddings). The default headless renderer is BROWSER_BACKENDS=cf-browser-rendering. Local ONNX fallbacks are disabled in the slim image; configure search and cloud retrieval through each authenticated subject's relay record. Worker-wide search/model chains and provider keys are not forwarded to the container.

The per-task [models.*] cells (in config.toml) configure cloud embed/rerank/chat: point each cell's base_url at any OpenAI-compatible endpoint — OpenRouter is the default — and supply the key via the cell's api_key or the matching HULL_<TASK>_API_KEY env. One OPENROUTER_API_KEY serves every cell that keeps the OpenRouter default base_url. Store keyed search-provider credentials (such as TAVILY_API_KEY) in the same subject record. Provision Vectorize and EMBEDDING_DIMS for a dimension supported by the selected embedding model.

Deployment (maintained instance)

Every tagged release deploys automatically (only while the CF_DEPLOY_ENABLED gate is on -- see the pause note below): the CD deploy-cf job checks out the released tag, builds the http-slim image, pushes it to the Cloudflare-managed registry as immutable :<release-tag>, deploys the Worker, and gates on a canary health check -- a release is live at exactly its own version. A beta dispatch redeploys the beta; a stable dispatch is maintainer-gated. Manual wrangler deploy against the maintained instance is not permitted: it would break the release-tag ↔ live-image correspondence. Self-hosting on your own Cloudflare account (the button above) is unaffected.

Paused 2026-09-13: the CF deploy token was removed from the account as off-manifest (process violation), so deploy-cf now no-ops behind the CF_DEPLOY_ENABLED repo variable. The maintained instance stays frozen at its last deployed release until a token is re-established via the documented process and the variable is set to true.

Smithery

wet ships a smithery.yaml so it can be installed and run through Smithery. The manifest declares a stdio start command (uvx --python 3.13 wet-mcp) with an empty config schema -- no config is required to start, and providers and credentials are configured at runtime via the server's own config flow (see Configuration).

Trust Model

This plugin implements TC-Local (machine-bound, single trust principal). See mcp-core trust model for full classification.

License

Apache-2.0 -- See LICENSE.