gocept.sftpcopy 3.0

Usage

You can either give the name of a configuration file on the commandline, or pass the configuration values as a dict directly to the entrypoint (useful for buildout integration). The configuration file has the following format:

[general]
mode = upload # or download
logfile = /path/to/logfile # defaults to stdout if not given
buffer_size = 65536
reconnect_attempts = 3
reconnect_delay = 1
skip_files =
    name_of_file_to_skip_1
    name_of_file_to_skip_2

[local]
path = /path/on/local/machine

[remote]
path = /path/on/remote/machine
hostname = remote.host
port = 22
username = user
password = secret

The configdict uses the following keys instead:

• logfile

• buffer_size (default: 65536, i.e. 64 KiB)

• keepalive_interval (default: 5 seconds)

• reconnect_attempts (default: 3)

• reconnect_delay (default: 1 second)

• local_path

• remote_path

• hostname

• port

• username

• password

• key_filename

• skip_files

key_filename takes precedence over password. If key_filename ends with dsa, it’s assumed to be a DSA key, else an RSA key. Note that the key file must not be password protected.

skip_files is a list of filenames (local or remote), which are skipped during upload or download.

Files are copied in chunks of buffer_size to avoid loading big files into memory at once.

If the SSH/SFTP connection drops while a file is being downloaded (e.g. with a paramiko.SSHException: Server connection dropped error), sftpcopy automatically reconnects, re-authenticates and resumes the download at the byte offset it had already written locally, instead of starting over. This is retried up to reconnect_attempts times (waiting reconnect_delay seconds in between); if the remote file changed size (or, where reported by the server, modification time) in the meantime, resuming is refused and an error is raised instead of assembling a corrupted file. Uploads are not resumable.

You can also use sftpcopy as a python object like this:

import gocept.sftpcopy
sftp = gocept.sftpcopy.SFTPCopy(
    '/path/on/local/machine',
    'remote.host', 22, 'user', 'secret', '/path/on/remote/machine',
    skip_files=['my_file_to_ignore'])
sftp.connect()
sftp.uploadNewFiles()  # or sftp.downloadNewFiles()

Changelog

3.0 (2026-08-28)

• Automatically reconnect and resume interrupted SFTP downloads.

2.1 (2025-07-16)

• Add compatability to Python 3.11 and 3.12.

2.0 (2020-06-18)

0.6.0 (2016-11-02)

• Pinning version numbers of dependencies for tests.

• Migrate to py.test as testrunner.

• Add new skip_files parameter to skip upload or download of files.

0.5.1 (2015-04-15)

• Update bootstrap.py to version from zc.buildout 2.3.0.

• Move repository to bitbucket.org.

0.5.0 (2014-11-26)

• Set up keep-alive checking.

0.4.1 (2014-03-07)

• Fixed brown-bag release.

0.4.0 (2014-03-07)

• Copy files in chunks instead of loading each complete file into memory. There is a new config option buffer_size which defaults to 64 kB.

0.3.0 (2014-02-20)

• Support key-based authentication.

• Check that the size of the transmitted file after up/download matches what was transferred.

0.2.2 (2012-04-04)

• Update to gocept.amqprun-0.8.

0.2.1 (2012-03-29)

• Make amqp server configurable for tests.

• Clean up garbage connections left by tests (#10634).

0.2 (2012-02-22)

• Add gocept.amqprun integration.

0.1.4 (2009-11-16)

• Log errors that occur while connecting

0.1.3 (2008-02-27)

• Added configdict argument to main function for easier buildout integration.

0.1.2 (2008-02-18)

• Fixed bug in connection logging.

• Remember filestore so we can actually upload/download.

• Did some testing predefined user.

0.1.1 (2007-11-13)

• Fixed brown back release 0.1 which was not usable at all since there were various files missing in the archive.