Terms Cockpit

PyPI Version Python Versions License

Terms Cockpit is a Python package for accessing, indexing, and analysing Terms of Service and Privacy Policy documents from OpenTermsArchive and ToS;DR repositories.

It ships both a reusable Python library and an optional web application with a full REST API, interactive document viewer, version history explorer, and UNFAIR_TOS clause classifier.

Features
Installation
Python Library
Web Server
Architecture
License
Acknowledgements

Features

Multi-repository support — load any number of OpenTermsArchive or ToS;DR snapshot repositories simultaneously.
Efficient change indexing — a single git log --numstat pass populates a SQLite database; subsequent starts are incremental.
Document retrieval — fetch latest content or any historical version by commit hash.
Readability extraction — strip navigation chrome from HTML with Mozilla Readability.
Version diffing — unified diffs of raw HTML or plain-text (tag-stripped) content between any two adjacent versions.
Text-change detection — identify which versions have actual visible-text changes, ignoring formatting-only commits.
UNFAIR_TOS classification — stream per-paragraph clause analysis via Server-Sent Events using fine-tuned LegalBERT, zero-shot NLI, or cosine-similarity backends.
Interactive web UI — service browser, document viewer, version history chart, diff viewer, evolution sparklines, and LexGlue analysis page.
OpenAPI / Swagger UI — auto-generated API documentation at /api/docs.

Installation

# Core library only
pip install termscockpit

# Core + web server
pip install 'termscockpit[server]'

# Core + web server + LexGlue classifier
pip install 'termscockpit[server,lexglue]'

# From source
pip install git+https://github.com/cruzlorite/termscockpit.git

Python Library

All business logic is available as a standalone Python package, independent of the web server.

TermsCockpit

The main entry point. Clones (or pulls) a remote snapshot repository, enumerates services and documents, and provides content and history query methods.

from termscockpit import TermsCockpit

# Uses the OpenTermsArchive Community repo by default
tos = TermsCockpit(track_changes=True)

# Enumerate
print(tos.services)                        # ['Google', 'Facebook', ...]
print(tos.documents['Google'])             # ['Google/Privacy Policy.html', ...]
print(tos.list_all_documents())            # flat list of all document paths

# Latest content
html = tos.get_document_content('Google/Privacy Policy.html')

# Content at a specific commit
html = tos.get_document_content('Google/Privacy Policy.html', commit_hash='abc123')

# Readability-extracted content
result = tos.get_document_readability('Google/Privacy Policy.html')
# {'title': '...', 'short_title': '...', 'content': '<html>...'}

# Version history
changes = tos.list_document_changes('Google/Privacy Policy.html')
# [(commit_hash, author, timestamp, insertions, deletions, blob_sha), ...]

# Summaries for a list of documents (batch-efficient)
summaries = tos.list_document_summaries(['Google/Privacy Policy.html'])
# {'Google/Privacy Policy.html': (version_count, last_timestamp)}

# Unified diff — raw HTML
diff = tos.diff_document_between_commits('Google/Privacy Policy.html', idx=5)

# Unified diff — visible text only (tags stripped)
diff = tos.diff_document_text('Google/Privacy Policy.html', idx=5)

# Indices of versions with actual text-content changes
indices = tos.list_text_change_versions('Google/Privacy Policy.html')
# [1, 3, 7, ...]

tos.close()

Constructor parameters

Parameter	Type	Default	Description
`track_changes`	`bool`	`True`	Build a SQLite change index on startup. Disable for faster startup when history is not needed.
`repo_url`	`str`	OTA Community	URL of the Git repository to clone.

Class constants

Constant	Value
`TermsCockpit.OTA_REPO_URL`	`https://github.com/OpenTermsArchive/contrib-snapshots.git`
`TermsCockpit.TOSDR_REPO_URL`	`https://github.com/tosdr/tosdr-snapshots.git`

GitChangeIndex

Low-level, reusable SQLite index over any Git repository. Can be used independently of TermsCockpit.

from termscockpit import GitChangeIndex
from pathlib import Path

index = GitChangeIndex(
    repo_path=Path('/path/to/repo'),
    index_path=Path('/path/to/index.db'),
)

# Per-file summary (batch query)
summary = index.list_files_summary(['service/doc.html'])
# {'service/doc.html': (version_count, last_timestamp)}

# Full history for one file
changes = index.list_file_changes('service/doc.html')
# [(commit_hash, author, timestamp, insertions, deletions, blob_sha), ...]

# File content at a commit
content = index.get_file_content('abc123def', 'service/doc.html')

# Unified diff between adjacent versions
diff_lines = index.diff_adjacent('service/doc.html', idx=3)

index.close()

The index is built with a single git log --numstat subprocess (orders of magnitude faster than per-commit GitPython API calls) and updated incrementally on subsequent runs.

document_utils

Pure-Python HTML utilities, usable without the server.

from termscockpit.document_utils import html_to_text, text_hash, apply_readability

# Strip tags and normalise whitespace
text = html_to_text(html)

# MD5 fingerprint of visible text (useful for change detection)
h = text_hash(html)

# Mozilla Readability extraction
result = apply_readability(html)
# {'title': '...', 'short_title': '...', 'content': '<html>...'}

lexglue

UNFAIR_TOS classification engine, usable without the server.

from termscockpit.lexglue import (
    UNFAIR_CATEGORIES, FAIR_LABEL, DEFAULT_MODEL,
    detect_backend, load_pipeline, score_batch,
    extract_blocks_annotated,
)

# Extract annotated segments from an HTML document
segments, annotated_html = extract_blocks_annotated(html, readability=True)

# Load inference pipeline (cached after first call)
pipe = load_pipeline('Agreemind/lexglue-legalbert-unfair-tos')

# Score a batch of text segments
scores = score_batch(pipe, 'Agreemind/lexglue-legalbert-unfair-tos', segments)
# [{'limitation of liability': 0.82, 'fair and unproblematic clause': 0.18, ...}, ...]

Supported backends (auto-detected from model config):

Backend	Description	Example model
`clf`	Fine-tuned multi-label classifier	`Agreemind/lexglue-legalbert-unfair-tos`
`nli`	Zero-shot NLI via entailment	`cross-encoder/nli-deberta-v3-small`
`sim`	Cosine similarity on CLS embeddings	`nlpaueb/legal-bert-base-uncased`

UNFAIR_TOS categories

Limitation of liability
Unilateral changes to the terms
Content removal by the provider
Jurisdiction clause
Choice of law
Mandatory arbitration
Unilateral termination by the provider
Contract by using the service

Web Server

Running the server

# Load all known repositories (clones on first run, ~several GB)
python -m termscockpit.server

# Load a subset of repositories
python -m termscockpit.server --repos contrib genai-contrib tosdr

# Faster startup — disable change tracking (history features unavailable)
python -m termscockpit.server --repos contrib --no-changes

# Custom host and port
python -m termscockpit.server --host 0.0.0.0 --port 8080

# Set the default active repository
python -m termscockpit.server --repo tosdr --repos tosdr contrib

The server starts immediately and loads repositories in the background. The UI shows each collection's loading progress; pages for a collection become interactive as soon as that repository is ready.

CLI options

Option	Default	Description
`--host`	`127.0.0.1`	Bind address
`--port`	`5000`	Bind port
`--repo`	`contrib`	Default active repository name
`--repos`	all	Whitespace-separated list of repository names to load
`--no-changes`	off	Disable git change indexing (faster startup)
`--debug`	off	Enable Flask debug mode

Known repositories

Name	Label	Group
`contrib`	Community	OpenTermsArchive
`genai-contrib`	GenAI Community	OpenTermsArchive
`india`	India	OpenTermsArchive — Regions
`kenya`	Kenya	OpenTermsArchive — Regions
`cote-divoire`	Côte d'Ivoire	OpenTermsArchive — Regions
`dating`	Dating	OpenTermsArchive — Topics
`p2b-compliance`	Platform-to-Business Compliance	OpenTermsArchive — Topics
`pga`	Professional Gaming	OpenTermsArchive — Topics
`dsa-reports`	DSA Reports	OpenTermsArchive — Topics
`genai-eu`	GenAI EU	OpenTermsArchive — Topics
`demo`	Demo	OpenTermsArchive — Other
`sandbox`	Sandbox	OpenTermsArchive — Other
`tosdr`	ToS;DR Snapshots	ToS;DR

REST API

Interactive API documentation (Swagger UI) is available at /api/docs when the server is running.

Base URL: /api/<repo>/

Services

Method	Path	Description
`GET`	`/api/<repo>/services/`	List all services
`GET`	`/api/<repo>/services/<service>/documents`	List documents for a service (with version counts and last-modified dates)

Documents

Method	Path	Description
`GET`	`/api/<repo>/documents/<path>`	Latest raw HTML content
`GET`	`/api/<repo>/documents/<path>/at/<commit>`	Raw HTML at a specific commit
`GET`	`/api/<repo>/documents/<path>/readability`	Latest readability-extracted content
`GET`	`/api/<repo>/documents/<path>/readability/at/<commit>`	Readability content at a specific commit
`GET`	`/api/<repo>/documents/<path>/changes`	Full version history
`GET`	`/api/<repo>/documents/<path>/diff/<idx>`	Unified diff at version index (`?text=1` for plain-text diff)
`GET`	`/api/<repo>/documents/<path>/text-change-versions`	Indices of versions with actual text changes

Repositories

Method	Path	Description
`GET`	`/api/repos/`	List all known repositories and their loading status
`POST`	`/api/repos/switch`	Switch active repository (body: `{"name": "<repo>"}`)
`GET`	`/api/repos/status/<name>`	Poll loading status for a specific repository

LexGlue Analysis

Method	Path	Description
`GET`	`/api/<repo>/lexglue/<path>/analyze`	Stream UNFAIR_TOS analysis via Server-Sent Events

Query parameters for the analyze endpoint:

Parameter	Default	Description
`model`	`Agreemind/lexglue-legalbert-unfair-tos`	HuggingFace model identifier
`commit`	latest	Analyse a specific document version
`readability`	`1`	Strip navigation chrome before analysis

SSE events: html, status, paragraph, error, done.

Architecture

termscockpit/
├── termscockpit.py       # TermsCockpit — main library class
├── git_change_index.py   # GitChangeIndex — SQLite-backed git history index
├── document_utils.py     # HTML utilities (html_to_text, text_hash, apply_readability)
├── lexglue.py            # UNFAIR_TOS classification engine (clf / nli / sim backends)
└── server/
    ├── app.py            # Flask app factory, multi-repo background loading
    ├── views.py          # Page routes (thin — serve templates only)
    ├── api.py            # REST API blueprints (thin wrappers over the library)
    ├── lexglue_api.py    # SSE streaming endpoint (thin wrapper over lexglue.py)
    ├── repos_api.py      # Repository management endpoints
    ├── __main__.py       # CLI entry point
    ├── static/
    │   ├── css/style.css
    │   └── js/app.js
    └── templates/
        ├── base.html
        ├── index.html      # Services listing + collection switcher
        ├── service.html    # Documents listing for a service
        ├── document.html   # Document viewer with version history
        ├── versions.html   # Version history chart + diff viewer
        ├── evolution.html  # Service-level document evolution
        └── lexglue.html    # UNFAIR_TOS analysis

Design principles:

The server API is a thin serialisation layer — all business logic lives in the Python package and can be used programmatically without Flask.
Repositories load in a background thread; the server accepts requests immediately and returns 503 for collections that are not yet ready.
The SQLite change index is built with a single subprocess call (git log --numstat) and updated incrementally, making restarts fast even for repositories with hundreds of thousands of commits.

License

This project is licensed under the MIT License.

Acknowledgements

Special thanks to ToS;DR and Open Terms Archive for maintaining the document repositories that power this tool.

License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python :: 3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

termscockpit-1.0.3.tar.gz (66.9 kB view details)

Uploaded Aug 21, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

termscockpit-1.0.3-py3-none-any.whl (76.3 kB view details)

Uploaded Aug 21, 2026 Python 3

File details

Details for the file termscockpit-1.0.3.tar.gz.

File metadata

Download URL: termscockpit-1.0.3.tar.gz
Upload date: Aug 21, 2026
Size: 66.9 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for termscockpit-1.0.3.tar.gz
Algorithm	Hash digest
SHA256	`2721025761b831706053bc8256bd054bf079bee83a6d2412d8310ab5e0965f16`
MD5	`7656129a3e4d98973f602e919fd42d4f`
BLAKE2b-256	`4868a8b83f7b1329ad2c6de475366cb56e637624f548b4d81b722a548a96c66a`

See more details on using hashes here.

Provenance

The following attestation bundles were made for termscockpit-1.0.3.tar.gz:

Publisher: publish-pypi.yaml on cruzlorite/termscockpit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: termscockpit-1.0.3.tar.gz
- Subject digest: 2721025761b831706053bc8256bd054bf079bee83a6d2412d8310ab5e0965f16
- Sigstore transparency entry: 2549023129
- Sigstore integration time: Aug 21, 2026
Source repository:
- Permalink: cruzlorite/termscockpit@85c73d37ef4084940825f2fcd018073b08d7674f
- Branch / Tag: refs/tags/v1.0.3
- Owner: https://github.com/cruzlorite
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish-pypi.yaml@85c73d37ef4084940825f2fcd018073b08d7674f
- Trigger Event: push

File details

Details for the file termscockpit-1.0.3-py3-none-any.whl.

File metadata

Download URL: termscockpit-1.0.3-py3-none-any.whl
Upload date: Aug 21, 2026
Size: 76.3 kB
Tags: Python 3
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for termscockpit-1.0.3-py3-none-any.whl
Algorithm	Hash digest
SHA256	`6e543581c05b7d6d1fd03d331f5d6dfd8751b2b30c1a3dc9f44da65b8bceb5d7`
MD5	`72dbde7e27921bfb16fed5ebd83f3a38`
BLAKE2b-256	`b372ab2456e28f3e0d3ef94d7ed736837477dc2ea0c8e121044401e8b42768c9`

See more details on using hashes here.

Provenance

The following attestation bundles were made for termscockpit-1.0.3-py3-none-any.whl:

Publisher: publish-pypi.yaml on cruzlorite/termscockpit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: termscockpit-1.0.3-py3-none-any.whl
- Subject digest: 6e543581c05b7d6d1fd03d331f5d6dfd8751b2b30c1a3dc9f44da65b8bceb5d7
- Sigstore transparency entry: 2549023186
- Sigstore integration time: Aug 21, 2026
Source repository:
- Permalink: cruzlorite/termscockpit@85c73d37ef4084940825f2fcd018073b08d7674f
- Branch / Tag: refs/tags/v1.0.3
- Owner: https://github.com/cruzlorite
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish-pypi.yaml@85c73d37ef4084940825f2fcd018073b08d7674f
- Trigger Event: push

Release history Release notifications | RSS feed

This release

1.0.3 This release

Aug 21, 2026 2 files

termscockpit 1.0.3

Terms Cockpit

Table of Contents

Features

Installation

Python Library

TermsCockpit

GitChangeIndex

document_utils

lexglue

Web Server

Running the server

CLI options

Known repositories

REST API

Services

Documents

Repositories

LexGlue Analysis

Architecture

License

Acknowledgements

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License

Requires

Provides Extra

Classifiers

Download files

Source Distribution

Built Distribution

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed