cipo-patent-data

Download the Canadian Intellectual Property Office's bulk patent data from its IP Horizons SFTP server, with a manifest that records, for every file, its size, SHA-256, retrieval time, source and a zip integrity test — so a download can be verified and cited later.

Unofficial tool. Not affiliated with or endorsed by the Canadian Intellectual Property Office. The MIT licence covers this code only. The data you download, and anything derived from it, is governed by CIPO's terms and conditions for the use of Canadian intellectual property data — read them before redistributing anything.

Why SFTP

CIPO publishes the same collections over public HTTPS, with no login. The SFTP route adds two things that matter for research data: a real directory listing (the HTTPS index is not browsable, so the file list otherwise has to be scraped), and a reference against which to check completeness — archives served over HTTPS have been found truncated. compare uses the listing to tell you whether files you already hold are the same as the server's, shorter, or a different size.

Install

pip install cipo-patent-data

Python 3.10 or newer, on Windows, macOS or Linux. The only dependency is paramiko.

1. Get an account and store the login

SFTP access is issued by CIPO on request. Register through the "Register for automated downloading (SFTP)" form on CIPO's IP Horizons: Download intellectual property data page; questions about access go to CIPO's IP data team at cipoipdata-opicdonneespi@ised-isde.gc.ca. CIPO sends a user name and a password. (CIPO's own SFTP guide describes WinSCP; its host key and folder names are out of date — see Host key below.)

Create a credentials file from credentials.env.example:

CIPO_SFTP_USER=the-user-name-cipo-sent-you
CIPO_SFTP_PASSWORD=

• Leave the password empty to be asked for it once per command (reconnects reuse it in memory).
• Or keep it out of files entirely: set CIPO_SFTP_USER and CIPO_SFTP_PASSWORD in the environment.
• Put the file elsewhere with CIPO_CREDENTIALS=<path> or --credentials <path>.
• On macOS and Linux, chmod 600 the file.

Never paste your login into an issue or a log excerpt. Nothing this tool prints contains the password. See SECURITY.md.

2. Choose where data goes

Downloads, the manifest and reports go to --dest, which defaults to $CIPO_DATA_DIR, or else a cipo-data folder in the current directory:

export CIPO_DATA_DIR=/data/cipo          # macOS / Linux
setx CIPO_DATA_DIR "E:\research\cipo"     # Windows (new terminals)

3. First run

Connect sparingly (see Connection limits below). A good first session:

cipo-patents dirs                  # one connection: what can this account see?
cipo-patents fetch --limit 3       # one connection: list the patent collections, fetch three files
cipo-patents status                # no connection

Then take what you need:

cipo-patents fetch --directory Patents_ST96_Weekly_Collection
cipo-patents fetch --directory Patent_CSV_2024_10_11 --match PT_main --match PT_interested_party
cipo-patents verify --deep         # no connection: re-hash and CRC-test every archive

Directory names change with each production run, so always start from dirs rather than a name written down somewhere — including in CIPO's guide, whose names are out of date. list, plan and fetch select the patent collections by default; --all-products adds trademarks, designs, images and sequence listings.

Commands

fetch options: --directory (repeatable), --match TEXT (repeatable), --limit N, --recheck, --attempts N, and --only-new to take only files compare marked as held nowhere else.

What fetch downloads: files not yet held, files marked failed, files whose local copy is missing, and files the server now lists at a different size (changed). A file that downloaded in full but failed its archive test is marked corrupt and is not downloaded again by a plain fetch — CIPO has been seen serving archives damaged at source, and re-fetching gigabytes on every run would not help. Retry one deliberately with fetch --recheck --match <name>.

Is this the same data I already have?

cipo-patents list
cipo-patents local-inventory --root /path/to/existing/cipo/archives
cipo-patents compare
cipo-patents fetch --only-new

compare puts every file into one of: only on the server; held but truncated locally (take the server copy); same name, different size; same name and size; or held locally but not in the listing you took. That last group includes older editions, and also any collection your list did not cover (by default list covers the patent collections only; use list --all-products to compare everything).

local-inventory never opens cloud-storage placeholders (OneDrive, iCloud Drive and similar), because opening one downloads it: they are reported as cloud-only with their size. Placeholders are detected on Windows and macOS; elsewhere they cannot be detected and would be read.

Files downloaded by hand

If you use WinSCP or FileZilla instead, run cipo-patents list once, save files anywhere, then:

cipo-patents ingest                # scans <dest>/raw and ~/Downloads, recursively
cipo-patents ingest --from /path/to/downloads --move

Each file's size is checked against the server listing (a short download is reported and not filed), the zip is tested, the file is hashed and copied into raw/<directory>/, and the row is marked source = manual. A file already saved in the right folder is adopted in place (renamed to the server's spelling if only the letter case differs).

With --move, an original is deleted only when an identical copy is filed: either the copy just made matched the original's SHA-256, or the filed copy was re-hashed and matched. If the filed copy has been damaged, the good original is filed over it instead. Every deletion is printed.

Host key

Before sending your login the tool checks the server's host key and refuses to connect if it does not match the pinned value.

The fingerprint in CIPO's guide is out of date. Guide v2.0 prints ssh-ed25519 256 wOX1WxRWVdy096BQNLi+T2VXtRLIvs1aS160SR3CguA=; when last measured (2026-09-16) the server offered no ed25519 key at all, only a 2048-bit RSA key:

ssh-rsa 2048 SHA256:v5YTto75n4lEn8uflIUH0aZfOxpUq0dlidXT/qpif2E

That RSA key is what this tool pins. It was observed, not confirmed by CIPO.

If CIPO changes the key you will get a mismatch error and no login is sent. Confirm the new fingerprint with CIPO, then set it:

CIPO_SFTP_HOSTKEY=ssh-rsa 2048 SHA256:<new fingerprint>

Connection limits

The server refuses an address for a while — about 40 minutes when observed — after several connections in quick succession. Each command opens one session. fetch reconnects only when a transfer drops, waits between attempts (30 s, then longer), and stops after eight reconnects in one run; running the same command again later resumes where it stopped. Avoid scripting loops of dirs or list.

Large transfers used to drop during SSH key renegotiation; the tool suppresses rekeying for the session and reads ahead in 64 MB windows, which keeps multi-gigabyte files moving.

Layout

<dest>/
  raw/<server directory>/...   downloaded archives
      *.part, *.part.json      an interrupted download and the server version it belongs to
  compare/                     server_inventory.csv, local_inventory.csv, comparison.csv
  manifest.csv                 one row per server file (see below)
  manifest.pending.csv         only if a save could not replace a locked manifest; read automatically
  .cipo-patents.lock           present while a command is running

manifest.csv columns: relpath, directory, filename, product, cadence, remote_size, remote_mtime, status, source, bytes, sha256, retrieved_at_utc, archive_test, tested_at_utc, note. Statuses: planned, downloaded, corrupt, failed, missing, changed, withdrawn. The manifest never contains credentials.

changed covers both a local copy that no longer matches its record and a newer version listed on the server; verify keeps the second visible until fetch has taken the new version, and it restores a row to downloaded only after re-hashing it (never under --no-rehash).

Before sharing or depositing the folder, leave out compare/ (its reports contain absolute paths from your machine), any *.part and *.part.json files (incomplete downloads), and the lock file. Keep the credentials file out of the data folder altogether.

Environment variables

Development

git clone https://github.com/maliyuam/cipo-patent-data
cd cipo-patent-data
pip install -e ".[test]"
pytest

The test suite runs entirely offline. Please do not add tests that connect to CIPO's server.

Citation

See CITATION.cff. Cite CIPO's data separately, according to CIPO's terms.

Licence

MIT, for the code. See LICENSE.