GTFS Scorecard

GTFS Scorecard

Daily, plain-language GTFS quality scorecards for small transit agencies: see the current grade, the three things to fix, and why each one matters.

GTFS Scorecard is a daily data-quality service for small transit agencies and the program staff, maintainers, and vendors who support them. It publishes dated grades, prioritized fixes, history, program views, practitioner tools, and open data. It does not implement a competing GTFS validator. Correctness findings come from MobilityData's canonical validator.

Quickstart

Requires Python 3.12+, uv, and Java 17+ (the validator jar is downloaded automatically on first run).

cd pipeline
uv sync
uv run scorecard run --all

This fetches today's snapshot of every current configured feed, validates and scores it, and writes artifacts to data/artifacts/<agency>/<date>.json plus a latest.json and a cross-agency index.json. Retired aliases are excluded from the batch; an explicit historical rescore writes a dated record without recreating mutable current files. Re-running a day is idempotent. Checks (from the repo root; mirrors the CI gate):

make verify

Run the web app locally

The frontend reads the JSON artifacts over HTTP. Serve the repo root and open the page through http://, not by double-clicking the file:

cd ..            # repo root, so data/artifacts/ is reachable
python3 -m http.server 8000
# then open http://localhost:8000/web/index.html

Opening web/index.html as a file:// URL leaves the page stuck on "Loading scorecards…": browsers block ES module loading and fetch over file://, so the app never runs. Any static server works; the only requirement is that data/artifacts/ sits one level above web/, which the ../data/artifacts fallback in web/src/app.js expects.

Where it fits

GTFS Scorecard is the evidence and triage layer:

1. Mobility Database and Transitland help identify public feeds and their sources or versions.
2. The MobilityData GTFS validator supplies canonical specification findings.
3. An agency, vendor, editor, or support program changes and publishes the feed through its existing workflow.
4. A 90-day GTFS Scorecard pilot is testing the final handoff: link an accepted action to the intended feed, run a comparable recheck, and preserve the result.

The California GTFS Quality Dashboard inspired the project's daily scorecard and support-program framing. GTFS Scorecard is independent and is not an official compliance determination.

Pilot agencies: Unitrans (ASUCD / City of Davis) and Yolobus (Yolo County Transportation District). Beyond the pilots, the registry contains more than 2,600 curated feed records, mostly in the United States and Canada, plus reviewed canaries across Europe, Asia-Pacific, and South America. Scoring is scheduled daily for every registered feed, and the public status page reports the exact current configured and published counts along with when the latest run completed. Read the published total there rather than from this repository: automation stopped committing generated data at the S3 cutover (see docs/follow-ups.md), so what git carries is a fallback snapshot that moves only when it is deliberately re-materialized from the live corpus (last refreshed 2026-08-07), with more than 2,100 numeric scorecards published. CI can only gate the snapshot figure, because make verify runs offline. A feed record is not always a distinct transit agency: regional feeds and modal variants are counted separately while the identity registry is reconciled. A record the Mobility Database has retired in favor of another is no longer one of them: it keeps its dated artifacts and redirects its old URL, and only its successor publishes a current grade, so one agency does not carry two (docs/feed-supersessions.md). That leaves the published count lower than it was, and right.

Live: gtfsscorecard.org — with the latest completed-run evidence at gtfsscorecard.org/status.

Status: Beta. The public scorecards and daily evidence service are live. The remediation workflow is being tested and is not yet a proven service. Realtime quality is scored only when a usable realtime feed is configured and measured; otherwise it is shown neutrally as not yet measured. Any agency can be added through registry/intake.yaml.

Use, adopt, or contribute

• Check a feed: browse gtfsscorecard.org, or request a one-off score.
• Track an agency: use the self-serve submission form, or follow the repository walkthrough to open a pull request.
• Gate a feed in CI: add the GTFS Scorecard Marketplace Action to a feed repository.
• Test what happens after the scorecard: read the remediation pilot request if you can take a finding through an accepted request and same-feed recheck.
• Contribute: choose a bounded open issue or read CONTRIBUTING.md. Feed corrections, accessibility review, and practitioner feedback are useful without changing scoring code.
• Get help, or fund the work: SUPPORT.md is the front door for both. Scoring is free for every agency and stays that way; the support page explains what running the service costs, what sponsorship would pay for, and what it never buys.
• Report on every agency you support: the program rollups (/program/<state>/) are free; the paid program report bundle packages a branded board report per agency into one archive.

What an agency gets

• An overall grade from the categories that can be measured: correctness, freshness, rider experience completeness, and, when available, realtime quality.
• "Top 3 things to fix", in plain language with effort hints. Findings are framed as fixes, never as failures.
• For U.S. agencies only, an NTD GTFS-readiness read (published, valid, current, and agency_id present) plus a neutral check of whether agency_id happens to equal the five-digit NTD ID. Equality is optional; RY2026 presence and the P-50 crosswalk are not.
• Trend history, one JSON artifact per agency per day.
• An embeddable grade badge (<agency>/badge.svg) the agency can put on its own developer page.

The scoring methodology, with citations to the California Transit Data Guidelines v4.0 and the validator's rule taxonomy, is in docs/rubric.md. Methodology changes are governed: a validator-version bump must attach the shadow-run impact report from scorecard canary before it lands (rubric.md, "Governed upgrades"). The grade, weights, thresholds, and fix order are GTFS Scorecard policy choices, not an official MobilityData grade or a universal quality standard. Feed sources and licenses are in docs/feeds.md. Forward planning is split in two: the infrastructure and scaling plan is in docs/roadmap.md, and what the product becomes for its users is in docs/product-roadmap.md.

What's on the site

Each agency gets a scorecard page, plus three companion pages written for the different seats at an agency check-in: a board one-pager (/agency/<id>/board/), a call-prep brief (/brief/), and, when comparable finding-clearance records exist, a clearance log (/fixes/). Around those sit:

• Coverage views — the coverage overview (/pulse/), most common problems (/problems/), realtime reliability (/realtime/), optional-feature adoption and accessibility data coverage (/adoption/), a consumer feature finder with translation-language and accessibility filters (/app/#/?view=features), and worldwide agency and route maps (/map/, /routes/). U.S.-specific NTD readiness (/ntd/) and equity (/equity/) stay available as regional modules.
• Program pages (/program/<state>/) for 46 states plus DC and named cohorts from rollups.yaml, each with the fixes shared across the group.
• Practitioner tools — request a one-off score through GitHub or run it locally (/try.html), check a feed before publishing (/check/), compare two agencies (/compare/), query the dataset with SQL in the browser (/query/), and put feed quality in a vendor contract (/procurement/).
• Board-ready outputs — each agency has a printable one-pager; see the live Unitrans example, or generate a self-contained, custom-branded offline report with the board-report tool.
• A fix knowledge base (/fix/<rule>/, one plain-language page per common validator finding) and the standards crosswalk (/crosswalk/).
• Machine-readable surfaces — the versioned read API (docs/api.md), a Parquet table for bulk SQL, per-agency badges and conformance marks, Atom change feeds, monthly citable dataset releases, and a read-only MCP server (docs/mcp.md).

The country controls describe only the current covered set. They are not a census of a country or region. The live evidence and readiness result for a bounded European GTFS beta are on the status page; the release gate and scope limits are in docs/global-expansion.md.

Use it in CI

Gate your own pipeline on feed quality with the published action. It scores the feed, prints the grade and the top fixes in the job log, and fails the build if the feed drops below min-grade or expires within min-days-to-expiry:

- uses: ChelseaKR/[email protected]
  with:
    feed-url: https://your-agency.example/google_transit.zip
    country: CA
    min-grade: C
    min-days-to-expiry: 14

Pin a full release tag or a commit SHA, as above. The floating major @v1 still exists for consumers who already use it, but it is not recommended here and these examples no longer use it: it is one mutable pointer that moves to the newest v1.x.y on release day, so it changes what your workflow runs without a commit or a diff on your side. v1.5.1 is the newest published Action release, and docs/ci-action.md says what it includes.

country is the feed's assigned ISO 3166-1 alpha-2 code; it defaults to US for existing workflows. Both thresholds are optional; leave one blank to skip that check. Full input reference and a complete workflow are in docs/ci-action.md.

Operating at scale

Beyond run, the CLI carries the commands the rollout plan needs:

scorecard sync --country US --state California   # propose keyless, untracked
                                                 # Mobility Database entries
scorecard shards --count 4                        # JSON fan-out plan for CI
scorecard reindex                                 # rebuild index.json from disk
scorecard rollups                                 # publish program rollup artifacts
scorecard render-site                             # crawlable static pages + sitemap
scorecard alerts --out digest.md                  # expiry/regression digest
scorecard notify                                  # per-subscriber digest (dry run)

Add --source-metadata-out <path> to scorecard sync for a source receipt that accounts for every recognized Mobility Database Schedule row as excluded, filtered, already tracked, duplicate, conflicted, or proposed for review. It never edits the registry and does not treat a catalog proposal as permission to republish.

notify builds a feed-health email for each opt-in subscriber in subscriptions.yaml, containing only the agencies they follow and only when one needs attention. It prints the emails by default; the daily workflow sends them via SES once an operator verifies a sender, applies the SES grant in infra/artifacts, and sets the SES_FROM repo variable.

The daily workflow fans agencies out across a parallel matrix and can mirror artifacts to a CloudFront-backed S3 bucket once infra/artifacts is applied (ADR 0002). Program rollups are configured in rollups.yaml as named cohorts (a liaison's own agencies, a district, the whole state) and shown at #/programs; an agency "needs attention" when its feed is expiring or its grade regressed, not merely when it scores below a B. The published JSON is a documented read API (docs/api.md); a flat catalog of every agency (grade, score, feed URL, days-to-expiry, top fix) is served at /catalog.json and /catalog.csv so a consumer needs one request, not one per agency.

Roadmap status: built vs deployed

The roadmap plans the path from two pilot feeds to a worldwide-capable service. The Year 1 software is built, tested, and mostly deployed; the deploy runbook walks through the AWS stacks and carries the current deployment status.

The cohort drafted from the Mobility Database has grown well past the first California pass: the manifest-backed registry now carries more than 2,600 curated feed records, mostly across the US and Canada, with a 760-record reviewed European cohort across 28 countries. It now includes a geographically diverse reviewed canary cohort, scored daily (a 2026-07 dedupe pass removed ~350 records that duplicated an already-listed feed).

Add your agency

Any agency with a public GTFS feed can be added with one YAML block in registry/intake.yaml and a pull request; the walkthrough is docs/add-your-agency.md. The live web form at web/submit.html does the same without YAML through the deployed infra/submit endpoint; every submission still opens a pull request for human review before publication.

What is free, and what is paid

Everything an agency needs for its own feed is free and stays free: its scorecard, the prioritized fixes, the history, the printable board one-pager, the self-contained board report it can hand to a board or attach to a grant application, the badge, the open data, and the read API. No account and no payment is involved anywhere in that path.

Two things cost money. A program that prepares packets for many agencies at once (a state DOT, a technical-assistance center, a feed vendor, a consultancy) can buy the program report bundle: a single archive holding a board report for every agency id it names, carrying the program's own name, logo, and accent color on each cover, refreshed monthly if it wants. A vendor, a consultancy, a program, or a researcher who needs the whole corpus over time can buy the scorecard history tables: every dated check and every finding as two Parquet tables with a data dictionary, provenance, and license, rebuilt monthly and licensed to one organization (ADR 0063, docs/history-tables.md). Plan sizes, prices, delivery terms, and the refund commitment live on the bundle page and the history tables page, which read them from the live plan data; they are not copied into this file, where they would go stale. A purchase buys no influence over grades, methodology, or which agencies are listed, and nothing was subtracted from the free tier to create either.

Support and sponsorship

The public scorecards, data, API, local pre-publish check, and request-backed one-off scoring remain free. The support page explains the project's infrastructure needs, what each GitHub Sponsors tier is sized to, and what sponsorship never buys; it does not advertise a payment rail that is not ready. Nothing is subtracted from the free tier to create anything else.

Standards conformance

This repo is governed by the shared portfolio standards vendored at docs/standards/ (pinned to a tag — never edited locally; see the integrity note in that directory's history). Per docs/standards/README.md's conformance rule, every applicable standard is declared here; none is silently skipped.

Open gaps per standard, as of the most recent conformance audit, are tracked in docs/standards-conformance-gaps.md rather than restated here.

Observability

Tier B (frontend) + Tier C (batch pipeline) — not the OBSERVABILITY-STANDARD's default Tier-A mapping for this repo's class; see ADR 0031 for why. In short: the deployed system is a scheduled Actions batch job plus a static site, not a long-lived hosted service (infra/compute, the piece that would make this Tier A, is built but not applied). Tier C's health/tracing/SLO controls are N/A (no network surface to probe). For Tier B, every fresh site build passes a blocking structural SEO check. Lighthouse gates code changes, and a weekly synthetic run checks representative production routes. Build-time SEO reports are retained for 14 days and production Lighthouse reports for 90 days. The public site carries one first-party measurement script, web/src/measure.js, with two separate blocks. The PostHog block (ADR 0055) sends page views and bundle checkout clicks to PostHog Cloud US with no cookie and no identifier that outlives the browser tab, and is off unless the deploy carries a POSTHOG_KEY. The Google Analytics 4 block (ADR 0056) loads GA4 for the measurement id in site-seo.json (measurement_ga4_id). GA4 records page views and engagement with the query string and referrer path dropped. It sets the first-party _ga cookies, which last up to two years, except in the EEA, the UK and Switzerland, where analytics storage is denied and no cookie is set. Google signals, ad personalization and ad storage are off everywhere. Both blocks load nothing under Global Privacy Control or Do Not Track, or for a reader who used the "Opt out of analytics" link in every page footer, which is remembered on that device and also deletes the GA4 cookies. What they record is stated at /about/#privacy and held by the structural check. Neither is real-user monitoring: no Core Web Vitals are collected. Search Console ownership verification and sitemap submission remain external domain-owner tasks; this repository stores no Search Console credentials or configuration. See the ADR and deploy runbook.

Versioning

SemVer. The public API surface is the published JSON schema (schema_version), the GitHub Action's inputs (action.yml), and the CLI (docs/api.md). pipeline/pyproject.toml's [project].version is the single source of truth; CITATION.cff and server.json mirror it, and pipeline/scripts/check_versions.py fails make verify if they drift. Supported-version policy: latest major only — this is a single-deployment civic tool, not a library with multiple consumers pinned to old majors.

This repo also produces releases: the marketplace GitHub Action (tagged v1) and monthly citable dataset releases (dataset-release.yml). It carries a publish workflow for the gtfs-scorecard Python package (pypi-publish.yml) and an MCP server manifest (server.json). gtfs-scorecard 1.5.1 is on PyPI, and the MCP Registry lists io.github.ChelseaKR/gtfs-scorecard 1.5.1 with that package. The v1.5.0 run of the workflow reached the upload under the package's former name scorecard-pipeline, which PyPI refused as too similar to an existing project, so nothing was uploaded for 1.5.0 and the distribution is gtfs-scorecard from 1.5.1 on. A release tag starts the PyPI workflow, which then waits for a reviewer to approve the upload; the manifest goes to the registry only after that upload exists.

uvx gtfs-scorecard starts the MCP server and uvx --from gtfs-scorecard scorecard runs the command-line tool; the import package stays scorecard_pipeline. To run code on main that is not released yet, install from the source tree instead:

uvx --from "git+https://github.com/ChelseaKR/gtfs-scorecard#subdirectory=pipeline" scorecard-mcp

docs/mcp.md has the client config and the publishing steps.

Run your own instance

A state DOT, national RTAP, or country program can fork this repo and stand up its own branded instance — own domain, own agency registry, own organization name — from config and a deploy, no code change: docs/fork-quickstart.md.

Layout

registry/       manifest-backed agency shards; add yours to intake.yaml
instance.example.yaml   fork branding template; copy to instance.yaml to rebrand
rollups.yaml    program rollups (portfolio views across many agencies)
pipeline/       Python pipeline: fetch -> validate -> score -> publish
web/            static frontend; reads only published JSON
infra/          Terraform: artifacts CDN, submission function, fan-out compute
docs/           feeds, rubric, roadmap, api, fixes/ (KB), decisions/ (ADRs)
data/           raw snapshots (ignored) and published artifacts (committed)

Guardrails

Three rules hold everywhere in this project: every metric is published with a plain-language explanation of what it means; accessibility (WCAG 2.2 AAA) is non-negotiable in the web app; and an agency without a realtime feed is shown neutrally, never shamed. Contributor and agent-facing build rules live in CLAUDE.md; current direction beyond the shipped roadmap is worked in the open as labeled notes in docs/ideation/.

Maintainer

Built and maintained by Chelsea Kelly-Reif, starting from the transit systems in Davis, California. Contributions and corrections are welcome; agencies can ask to be added or removed under the listing and removal policy.