FrameFetch turns one YouTube, YouTube Shorts, TikTok, Instagram Reels, Pinterest, or Reddit video URL into a single JSON response: a direct answer to a question about the video, metadata, engagement insights, a transcript (captions or Whisper), an LLM digest (text or spoken mp3), structured JSON (chapters/entities/products/claims), , (every Nth / 1-per-second / a time range, at any width), and the (OCR — captions, price tags, signage). Plus keyword when you don't have a URL yet, and for up to 10 URLs in one call. Built API-first and MCP-first for AI agents.
Any social-video URL → transcript, metadata, frames, OCR, summary, search, Q&A. MCP server + x402.
2026-11-09
Stars
—
Tools
—
Uptime
—
comments + sentiment
parametrically-sampled frames
on-screen text burned into those frames
search
batch
This repo is the open-source client + docs. The service itself runs at framefetch.net — you bring a free API key (or pay per call with x402); the backend stays hosted.
Why
An LLM can't watch a video. To reason about one it needs the video turned into text and images first — an answer, a transcript, metadata, a few frames. FrameFetch returns all of that from a URL, across six platforms, through one schema.
Install
npm install framefetch
Node 18+ (uses built-in fetch). Get a free key: framefetch.net.
Version note. This repo is at 0.4.0. The newest version currently on npm is 0.3.0 — npm install framefetch still gives you that one, and it has only extract/metadata/transcript/frames/platforms/status/demo/createKey. Everything else documented below is live on the API today and available from this repo; from npm 0.3.0 you can reach the same data through extract({ fields: [...] }).
Ask a question — get an answer, not a transcript dump
A direct question about a video returns a short, grounded answer with timestamped quotes, instead of you parsing a 25,000-token transcript yourself.
import { FrameFetch } from 'framefetch';
const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });
const { ask } = await ff.ask(
'https://www.youtube.com/watch?v=jNQXAC9IVRw',
'What does the presenter say to do first?',
);
console.log(ask.answer); // short, direct answer
console.log(ask.confidence); // 'high' | 'medium' | 'low'
for (const q of ask.quotes) { // verbatim, timestamped supporting quotes
console.log(`[${q.t_sec}s] ${q.text}`);
}
console.log(ask.coverage); // which part of the transcript was analyzed
Charged only when an answer is actually produced. A repeat question about an already-extracted video reuses the cached transcript, so it answers fast without a re-download or re-transcription — but the answer itself is always freshly generated, never cache-served.
Frames-based answers: when a video has no transcript (e.g. Pinterest, or transcription failed), the answer is grounded in sampled keyframe images instead. Then coverage.mode is "frames", quotes is [] (no transcript text to quote), and confidence is capped at "medium".
Quick start
import { FrameFetch } from 'framefetch';
const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });
const r = await ff.extract({
url: 'https://www.youtube.com/watch?v=jNQXAC9IVRw',
fields: ['metadata', 'transcript', 'frames', 'text_overlay'],
frames: { mode: 'fps', fps: 1, width: 480 },
});
console.log(r.metadata.title); // "Me at the zoo"
console.log(r.transcript.text); // "All right, so here we are, in front of the elephants…"
console.log(r.frames.length); // 19 — frames is an array
console.log(r.textOverlay?.[0]?.text); // on-screen text detected in the first frame, if any
Note the two spellings: text_overlay is the request field name, textOverlay is the response key.
Scoped helpers
await ff.metadata(url); // title, author, duration, views, likes…
await ff.transcript(url); // captions, else Whisper
await ff.frames(url, { mode: 'fps', fps: 1, width: 512 });
await ff.ask(url, 'What product is being reviewed?'); // grounded Q&A, see above
await ff.digest(url); // LLM summary of the transcript
await ff.audioDigest(url, { voice: 'nova' }); // spoken mp3 briefing (signed URL, 24h)
await ff.structured(url); // chapters/entities/products/claims/key_moments
await ff.comments(url, { comments_cap: 50 }); // top-level comments
await ff.commentSentiment(url); // aggregated audience-mood rollup (+ the comments)
await ff.platforms(); // capability matrix (no key)
await ff.status(); // live service health (no key)
// on-screen text (OCR) — requires "frames" alongside it, use extract() directly:
await ff.extract({ url, fields: ['frames', 'text_overlay'], frames: { mode: 'fps', fps: 1 } });
Every helper above is a thin wrapper over extract(), so anything extract() accepts (translate,
format, extra fields, …) can be passed as the last argument and is forwarded unchanged.
Search for videos, extract many at once
// Find something to extract when you don't have a URL yet
const s = await ff.search('how to make sourdough', { limit: 5 });
for (const hit of s.results) {
console.log(hit.title, hit.url, hit.durationSec);
}
// Then extract up to 10 of them in ONE call. Shared options apply to every url.
const b = await ff.batch(s.results.slice(0, 3).map((r) => r.url), {
fields: ['metadata', 'digest'],
});
for (const item of b.results) {
if (!item.ok) { console.error(item.url, item.error?.code); continue; }
console.log(item.metadata.title, '→', item.digest.gist);
}
One failing URL never fails the batch — each entry carries its own ok flag and, when ok is false,
an error with code/message/hint. Per-URL frames specs are not accepted in a batch; use
extract() for those.
Translate the transcript, export subtitles
// translate the transcript into 1 of 25 languages (surfaced as transcript_translated)
const r = await ff.transcript(url, { translate: 'ja' });
console.log(r.transcript_translated.text);
// export subtitles directly — format is sent as a query param, response comes back as a string
const srt = await ff.transcript(url, { format: 'srt' }); // source-language subtitles
const vttJa = await ff.transcript(url, { translate: 'ja', format: 'vtt' }); // translated subtitles
FrameFetch ships an MCP server (Streamable HTTP) with four tools: framefetch_extract,
framefetch_platform_capabilities, framefetch_search and framefetch_account. Add it to Claude,
Cursor, or any MCP client:
MCP lives at https://framefetch.net/mcp and speaks JSON-RPC over Streamable HTTP. REST lives under
/v1/* and takes plain JSON ({"url": "…"}). Crossing the two is the single most common first-call
mistake, so both directions answer clearly: a REST body POSTed to /mcp comes back as a JSON-RPC
parse error, and a JSON-RPC body POSTed to /v1/extract comes back as 400 WRONG_ENDPOINT naming
the right URL for your client.
Local stdio bridge
Prefer a local stdio server (Claude Desktop, sandboxes, no inbound HTTP)? This package
ships framefetch-mcp, a zero-dependency stdio↔HTTP bridge that exposes the same tools
and forwards calls to framefetch.net:
tools/list works with no key; tool calls use FRAMEFETCH_API_KEY (or x402). Override the
endpoint with FRAMEFETCH_MCP_URL.
Pay without an account (x402)
Autonomous agents can pay per call in USDC via x402 on Base — no signup, no human in the loop. Discoverable in the x402 Bazaar and at /.well-known/x402.json. Humans can use a free tier, prepaid credits, or a Stripe card.
Errors
Failed calls throw FrameFetchError with .status, .code, and .hint:
import { FrameFetchError } from 'framefetch';
try {
await ff.transcript(url);
} catch (e) {
if (e instanceof FrameFetchError && e.status === 402) {
// out of credit — top up at framefetch.net or via x402
}
}
API surface
Method
Endpoint
Auth
extract({ url, fields, frames, … })
POST /v1/extract
key
ask(url, question)
POST /v1/extract (ask param)
key
metadata(url)
POST /v1/metadata
key
transcript(url, { translate, format })
POST /v1/transcript
key
frames(url, spec)
POST /v1/frames
key
digest(url)
POST /v1/extract (digest)
key
audioDigest(url, { voice })
POST /v1/extract (audio_digest)
key
structured(url)
POST /v1/extract (structured)
key
comments(url, { comments_cap })
POST /v1/extract (comments)
key
commentSentiment(url)
POST /v1/extract (comment_sentiment)
key
search(query, { limit })
POST /v1/search
key
batch(urls, { fields })
POST /v1/batch
key
platforms()
GET /v1/platforms
—
status()
GET /v1/status
—
demo(url)
POST /v1/demo
—
createKey(email)
POST /v1/keys
—
Full extract() request shape
ff.extract({
url: string,
fields?: Field[], // 'metadata' | 'insights' | 'transcript' | 'frames' | 'text_overlay'
// | 'digest' | 'audio_digest' | 'structured' | 'comments'
// | 'comment_sentiment' | 'delta'
frames?: { mode, n, fps, from, to, format, width },
translate?: string, // ISO-639-1 target language (25 supported)
voice?: string, // TTS voice for audio_digest
comments_cap?: number, // 1-200, default 100
ask?: string, // 3-500 char question — see ff.ask() above
publish?: boolean, // opt in to a public per-video SEO page
format?: 'md' | 'markdown' | 'srt' | 'vtt', // alternate egress rendering (returns a string, not JSON)
});
See index.d.ts for the complete typed response shape (ExtractResult, Ask,
VideoStructured, VideoComments, CommentSentiment, AudioDigest, SearchResult,
BatchResult, …).
