World Wide Law
Open-source collection scripts for open legal data from 110+ countries.
Every country publishes its laws, court decisions, and regulations online -- but in different formats, behind different APIs, with different access rules. World Wide Law is building the open infrastructure to collect, normalize, and make all of it searchable.
All sources in this repository are open data -- publicly available legal information from official government portals, APIs, and bulk download endpoints. We always prefer API and bulk access over web extraction.
Live Dashboard & API
- Dashboard: legaldatahunter.com -- track coverage, explore sources, submit feedback
- Search API: Available at legaldatahunter.com -- search across 16M+ indexed legal documents
What's Here
This repository contains 960+ collection scripts across 110+ countries that download and normalize open legal data from government portals worldwide. Each script follows a standard interface so that any developer can run, test, or improve it. Some sources are marked as blocked (CAPTCHA, IP restrictions, etc.) -- their scripts are included so developers can review and potentially contribute fixes.
sources/
FR/LegifranceCodes/ # French consolidated legal codes (API)
DE/GesetzeImInternet/ # German federal laws (bulk XML)
IT/NormattivaLegislation/ # Italian legislation (API)
ES/BOE/ # Spanish official gazette (API)
... (110+ countries)
Quick Start
# Clone the repo
git clone https://github.com/worldwidelaw/legal-sources.git
cd legal-sources
# Install dependencies
pip install -r requirements.txt
# Check project status
python runner.py status
# Test a specific source
python runner.py sample FR/LegifranceCodes
# See what needs work
python runner.py next
How It Works
Per-Source Structure
Every source lives in sources/{COUNTRY_CODE}/{SourceName}/ and contains:
Two Data Models
Legislation (mutable): Laws get amended. Same ID, new content. Strategy: upsert with version tracking.
Case law (immutable): Court decisions don't change after publication. Strategy: append-only with dedup.
Standard Output Schema
Every script normalizes documents to a common schema:
_id-- Unique identifier_source-- Source identifier (e.g.,FR/LegifranceCodes)_type--legislationorcase_lawtitle-- Document titletext-- Full text contentdate-- Publication or decision dateurl-- Link to the original source
Architecture
legal-sources/
manifest.yaml # Master inventory: all sources + status
runner.py # CLI: run, test, and manage collection scripts
common/ # Shared libraries
base_scraper.py Base class all scripts inherit from
http_client.py HTTP client with retries + caching
rate_limiter.py Token bucket rate limiter
storage.py JSONL storage with deduplication
validators.py Schema validation
templates/ # Templates for new sources
scraper_template.py Boilerplate for bootstrap.py
config_template.yaml Boilerplate for config.yaml
retrieve_template.py Boilerplate for retrieve.py
sources/ # One directory per data source
{CC}/{Source}/ (see per-source structure above)
Coverage
Track live progress on the dashboard.
Contributing
We welcome contributions from developers, legal researchers, and especially governments who want their open legal data included.
Who can contribute?
Submit a data source (no coding required):
- Open a "New Source" issue and tell us about your country's legal data portal
Fix or improve a collection script:
- See CONTRIBUTING.md for the full guide
Report a problem:
- Data quality issue -- missing or incorrect data
- Bug report -- broken script
Good first issues: Browse label:good-first-issue for approachable starting points.
License
This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). You are free to use, modify, and distribute this software, provided that any modified versions made available over a network also make their source code available under the same license.
Commercial Licensing: If you wish to use this software without the AGPL-3.0 obligations (e.g., in a proprietary product or SaaS), commercial licenses are available. Contact [email protected] for details.