PhysioNet

A collection of tools for working with the PhysioNet repository.

Installation

pip install physionet

Requires Python 3.9 or later.

Command-Line Interface

The package provides a physionet command-line tool. You can also run it as a module with python -m physionet.

Find and inspect a dataset

Search published projects, inspect a specific version, then download the files you need:

physionet search "ECG" --resource-type database
physionet versions mimic-iv-demo
physionet show mimic-iv-demo --version 2.2
physionet download mimic-iv-demo --version 2.2 --include "*.csv"

Search results show the project slug, version, title, and access policy. show displays the license, DOI, size, topics, and a link to the PhysioNet page when available. Specify a version with show so the metadata refers to the exact dataset you intend to use. The PhysioNet page explains any access requirements; the server decides whether your account can download a project.

All three discovery commands support --json for scripts. JSON is written to stdout, and errors are written to stderr:

physionet search "ECG" --json > projects.json
physionet show mimic-iv-demo --version 2.2 --json > project.json
physionet versions mimic-iv-demo --json > versions.json

physionet download

Download datasets from PhysioNet:

# Download the latest version of a dataset
physionet download mimic-iv-demo

# Download a specific version
physionet download mimic-iv-demo --version 2.2

# Download to a specific directory
physionet download mimic-iv-demo --output /data

# Preview what would be downloaded
physionet download mimic-iv-demo --dry-run

# Download only specific files
physionet download mimic-iv-demo --include "*.csv" --exclude "*/notes/*"

Download sources:

The --source flag controls where files are downloaded from:

• auto (default) — tries S3 first, falls back to PhysioNet direct if the dataset is not available on S3
• physionet — always downloads from PhysioNet directly
• aws — downloads from S3 using boto3 and the standard AWS credential chain

# Download from PhysioNet directly
physionet download mimic-iv-demo --source physionet

# Download from S3 using boto3
physionet download mimic-iv-demo --source aws

When using --source aws, boto3 discovers credentials automatically via the standard AWS credential chain (environment variables, ~/.aws/credentials, IAM roles, etc.). This allows downloading credentialed datasets from S3 without passing PhysioNet credentials.

Authentication:

For credentialed datasets, provide PhysioNet credentials via flags or environment variables:

# Via flags
physionet download mimic-iv --username user --password pass

# Via environment variables
export PHYSIONET_USERNAME=user
export PHYSIONET_PASSWORD=pass
physionet download mimic-iv

PhysioNet credentials are sent only to the configured PhysioNet origin. Requests to S3 mirrors or other hosts do not receive them; --source aws uses the AWS credential chain described above.

Downloads support automatic resume, SHA256 checksum verification, and retry on transient errors.

If any selected file fails, the command reports the failed file and exits with status 1; successful files remain in the destination. In Python, download() raises DownloadError with the failed paths and causes. Pressing Ctrl-C exits the command with status 130 and leaves the incomplete file in place so a later HTTP download can resume it. If the server ignores the resume request, the client downloads the file from the beginning. Explicit AWS downloads currently restart incomplete files on the next attempt.

physionet validate

Validate a dataset before submission to PhysioNet. The validator checks for common issues that can delay the review process.

physionet validate /path/to/dataset

A validation report is automatically saved as PHYSIONET_REPORT.md in the dataset directory.

Options:

Check categories:

• filesystem - File naming issues (spaces, special characters, long names), proprietary formats (suggests open alternatives), hidden files, version control artifacts.
• documentation - Missing or incomplete documentation (e.g. README.md).
• integrity - CSV structure, encoding, and duplicate column detection.
• quality - Missing values, outliers, and data type consistency.
• privacy - PHI patterns (SSN, email, phone numbers), ages above 89, dates that may be real, and sensitive file detection.

Dates from the last 200 years up to today are reported as possibly real, since de-identified datasets usually shift dates (often into the future). This covers written dates in CSV and text files, and Unix timestamps in columns with time-like names such as charttime or timestamp. Dates shifted into that window still need to be confirmed by the submitter. Set ValidationConfig(date_lookback_years=...) to a number of years from 1 to 1000 to change the window, or None to turn the check off. The terminal shows date warnings for the first 10 files; the saved report lists every file.

CSV exports from the Empatica E4 wearable are recognized, so their headerless layout, empty tags.csv files, and Unix timestamps don't produce false errors or phone-number warnings. Their timestamps are checked as dates.

Examples:

# Run only filesystem and privacy checks
physionet validate /path/to/dataset --checks filesystem,privacy

# Save report as JSON to a custom path
physionet validate /path/to/dataset --report results.json

# Show only errors and warnings (suppress info messages)
physionet validate /path/to/dataset --level warning

# Scan all rows in CSV files (no sampling)
physionet validate /path/to/dataset --no-sampling

# Limit scanning to 5000 rows per file
physionet validate /path/to/dataset --max-rows 5000

Exit codes:

• 0 - Validation passed (no errors).
• 1 - Validation failed with errors.

Python API

Download

from physionet.download import download

# Download a dataset
download("mimic-iv-demo", version="2.2", output_dir="/data")

# Download from S3 using boto3
download("mimic-iv-demo", source="aws")

# Handle a partial failure while keeping successfully downloaded files
from physionet.download import DownloadError

try:
    download("mimic-iv-demo", version="2.2", output_dir="/data")
except DownloadError as error:
    for path, cause in error.failures:
        print(f"{path}: {cause}")

Validation

from physionet import validate_dataset, ValidationConfig

# Run with default settings
result = validate_dataset("/path/to/dataset")

# Run specific checks with custom settings
config = ValidationConfig(
    check_filesystem=True,
    check_documentation=True,
    check_integrity=False,
    check_quality=False,
    check_phi=True,
    max_rows_to_scan=5000,
)

result = validate_dataset("/path/to/dataset", config, show_progress=True)

# Print the summary report
print(result.summary())

# Export as a dictionary (for JSON serialization)
data = result.to_dict()

API Client

Interact with the PhysioNet REST API to explore and search published projects:

from physionet import PhysioNetClient

# Create a client instance
client = PhysioNetClient()

# List all published projects
projects = client.projects.list_published()
print(f"Total projects: {len(projects)}")

# Display first few projects
for project in projects[:5]:
    print(f"{project.slug} v{project.version}: {project.title}")

# Search for projects
ecg_projects = client.projects.search('ECG')
print(f"Found {len(ecg_projects)} ECG-related projects")
for project in ecg_projects:
    print(project.slug, project.version, project.access_policy, project.source_url)

# Get all versions of a project
versions = client.projects.list_versions('mimic-iv-demo')
for version in versions:
    print(f"Version {version.version}: {version.title}")

# Get detailed information about a specific version
details = client.projects.get_details('mimic-iv-demo', '2.2')
print(f"Title: {details.title}")
print(f"DOI: {details.doi}")
print(f"Published: {details.publish_datetime}")
print(f"Size: {details.main_storage_size} bytes")
print(f"Access policy: {details.access_policy}")
print(f"Topics: {details.topics}")

Authenticated Requests

For endpoints that require authentication (e.g., downloading checksums):

from physionet import PhysioNetClient

# Create client with authentication
client = PhysioNetClient(
    username='your_username',
    password='your_password'
)

# Download checksums file
client.projects.download_checksums(
    'mimic-iv-demo',
    '2.2',
    'checksums.txt'
)

# Or use environment variables
# Set PHYSIONET_USERNAME and PHYSIONET_PASSWORD
from physionet.api.utils import get_credentials_from_env

username, password = get_credentials_from_env()
client = PhysioNetClient(username=username, password=password)

Using Context Manager

from physionet import PhysioNetClient

# Automatically close session when done
with PhysioNetClient() as client:
    projects = client.projects.list_published()
    print(f"Found {len(projects)} projects")

Utility Functions

from physionet.api.utils import format_size

# Format bytes to human-readable size
size = format_size(16224447)
print(size)  # "15.47 MB"

Error Handling

from physionet import PhysioNetClient
from physionet.api.exceptions import NotFoundError, RateLimitError, ForbiddenError

client = PhysioNetClient()

try:
    details = client.projects.get_details('nonexistent-project', '1.0')
except NotFoundError:
    print("Project not found")
except RateLimitError:
    print("Rate limit exceeded, please wait before retrying")
except ForbiddenError:
    print("Access denied - check credentials or project permissions")

Contributing

Contributions are welcome!

License

This project is licensed under the MIT License - see the LICENSE file for details.