Skip to main content

A tool to copy sqlite databases you don't own

Project description

sqlite_backup

This exposes the python stdlib sqlite.backup function as a library, with a couple extra steps.

The main purpose for writing this is to copy sqlite databases that you may not own -- perhaps it belongs to an application (e.g., your browser) and is locked since that's currently open, or the OS keeps it open while the computer is active (e.g. Mac with iMessage)

Features

In short, this prioritizes safety of the data over performance (temporarily copying data files to /tmp) - because we often don't know what the application may be doing while we're copying underlying sqlite databases

The initial backup function and some tests were extracted out of the karlicoss/HPI core/sqlite module

If other tools exist to do this, please let me know!

Installation

Requires python3.7+

To install with pip, run:

pip install sqlite_backup

Usage

Usage: sqlite_backup [OPTIONS] SOURCE_DATABASE DESTINATION

  SOURCE_DATABASE is the database to copy

  DESTINATION is where to write the database. If a directory, uses
  the SOURCE_DATABASE name. If a file, the directory must exist,
  and the destination file must not already exist (to prevent
  possibly overwriting old data)

Options:
  --debug                         Increase log verbosity  [default: False]
  --wal-checkpoint / --no-wal-checkpoint
                                  After writing to the destination, run a
                                  checkpoint to truncate the WAL to zero bytes
                                  [default: wal-checkpoint]
  --copy-use-tempdir / --no-copy-use-tempdir
                                  Copy the source database files to a
                                  temporary directory, then connect to the
                                  copied files  [default: copy-use-tempdir]
  --copy-retry INTEGER            If the files change while copying to the
                                  temporary directory, retry <n> times
                                  [default: 100]
  --copy-retry-strict / --no-copy-retry-strict
                                  Throws an error if this fails to safely copy
                                  the database files --copy-retry times
                                  [default: copy-retry-strict]
  --help                          Show this message and exit.  [default:
                                  False]

For usage in python, use the sqlite_backup function, see the docs

If you plan on reading from these backed up databases (and you're not planning on modifying these at all), I would recommend using the mode=ro (readonly) or immutable flags when connecting to the database. In python, like:

import sqlite3
from typing import Iterator

def sqlite_connect(database: str) -> Iterator[sqlite3.Connection]:
    try:
        # or for immutable, f"file:{database}?immutable=1"
        with sqlite3.connect(f"file:{database}?mode=ro", uri=True) as conn:
            yield conn
    finally:
        conn.close()

with sqlite_connect("/path/to/database") as conn:
    conn.execute("...")

Example

sqlite_backup --debug ~/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite ./firefox.sqlite
[D 220202 13:00:32 core:110] Source database files: '['/home/sean/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite', '/home/sean/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite-wal']'
[D 220202 13:00:32 core:111] Temporary Destination database files: '['/tmp/tmpm2nhl1p3/places.sqlite', '/tmp/tmpm2nhl1p3/places.sqlite-wal']'
[D 220202 13:00:32 core:64] Copied from '/home/sean/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite' to '/tmp/tmpm2nhl1p3/places.sqlite' successfully; copied without file changing: True
[D 220202 13:00:32 core:64] Copied from '/home/sean/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite-wal' to '/tmp/tmpm2nhl1p3/places.sqlite-wal' successfully; copied without file changing: True
[D 220202 13:00:32 core:240] Running backup, from '/tmp/tmpm2nhl1p3/places.sqlite' to '/home/sean/Repos/sqlite_backup/firefox.sqlite'
Backed up /home/sean/.mozilla/firefox/ew9cqpqe.dev-edition-default/places.sqlite to /home/sean/Repos/sqlite_backup/firefox.sqlite

Tests

git clone 'https://github.com/seanbreckenridge/sqlite_backup'
cd ./sqlite_backup
pip install '.[testing]'
mypy ./sqlite_backup
pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sqlite_backup-0.1.7.tar.gz (14.4 kB view details)

Uploaded Source

Built Distribution

sqlite_backup-0.1.7-py3-none-any.whl (10.8 kB view details)

Uploaded Python 3

File details

Details for the file sqlite_backup-0.1.7.tar.gz.

File metadata

  • Download URL: sqlite_backup-0.1.7.tar.gz
  • Upload date:
  • Size: 14.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for sqlite_backup-0.1.7.tar.gz
Algorithm Hash digest
SHA256 81263978fe5ae5d0f389eef2976d7098ecf72bfb80702dc4ad2c59d53265ed6c
MD5 f81f02dfef9141c095616ecd46b80fe5
BLAKE2b-256 6271f474621612431e3e7ce35e24507fd3569b9b3fb89bb49ddc0ece22d88237

See more details on using hashes here.

File details

Details for the file sqlite_backup-0.1.7-py3-none-any.whl.

File metadata

File hashes

Hashes for sqlite_backup-0.1.7-py3-none-any.whl
Algorithm Hash digest
SHA256 9ba41e3c1acaafe2ed5242e83adfee4e2e2aa9abd32fbf59c1106793e260f5e9
MD5 0fd81176672b48a4c7ece6d7adfb4c04
BLAKE2b-256 80c4a6626e64db079f477ab7b52c9fc7412a673756d7ef4060315e15b555b0f0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page