Skip to main content

Filetracker caching file storage

Project description

filetracker

Build Status

A simple file storage module for distributed systems.

About

Filetracker has a client-server architecture: the server is the primary storage, and every client may have its own cache. Client has a Python API, and can be also invoked from the shell. Interaction between client and server is based on a simple HTTP API (Filetracker protocol).

Files are stored on the server compressed and deduplicated. A peculiar versioning scheme is supported: files are versioned by their modification timestamps, and some operations accept file versions as parameters (e.g. adding a file with an older version will have no effect if there's already a file with the same name and newer version).

Using

CAUTION: Filetracker has no security measures since it's meant to be used in internal networks. Don't put any sensitive data in filetracker without first making sure that it can't be reached from untrusted hosts.

Filetracker server requires Berkeley DB to run. On Debian-based systems it can be installed as libdb-dev.

After installing filetracker in a virtualenv, various scripts are added to $PATH. The most important ones are filetracker-server and filetracker. A simple filetracker server can be started with filetracker-server -L log.txt -l 127.0.0.1 -p 9999 -D. Read the scripts' help pages for more detailed information on running them.

Scripts

There are some useful scripts for working with filetracker in filetracker.scripts package, namely:

  • filetracker-migrate for uploading directories recursively to the server
  • filetracker-recover for recovering storage metadata after serious failures
  • filetracker-cache-cleaner for cleaning client cache

Check out the source or run them with -h for detailed information.

Upgrading from older versions

Testing

The recommended way to run tests is using tox. Install tox globally (either with pip install tox, or using your distribution's package manager), and simply run tox.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for filetracker, version 2.1.5
Filename, size File type Python version Upload date Hashes
Filename, size filetracker-2.1.5.tar.gz (37.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page