pyfastcopy
pyfastcopy is a simple Python module that monkey patches the shutil.copyfile function of Python standard library to internally use the sendfile system call.
It can provide important performance improvements for large file copy (typically 30-40%). See the performance section for some numbers.
Because shutil.copyfile is used by other fonctions in the shutil module, the following functions also automatically benefit from the performance boost:
shutil.copy
shutil.copy2
shutil.copytree
If sendfile is not available on your system or fails, the classic, slower chunk file copy is used, so there is no downside to using this module.
For more information, see my proposed patch for Python.
Performance
Tests were done copying files (source and destination) on a tmpfs filesystem, so that no slowdown related to hard drive or SSD storage occurs. Test files were generated with pseudo random data using frandom.
See benchmark.py for details about the test procedure and how the following graphs were generated.
Python 3.4: 
Python 3.6: 
These tests show a 30-50% performance improvement of ``shutil.copyfile`` compared to stock Python.
Usage
Just import the module:
import pyfastcopy
And then use the patched shutil.copyfile as usual:
shutil.copyfile(src, dst)
The ``sendfile`` system call does not exist on Windows, so importing this module will have no effect.
Installation
From PyPI (with PIP)
If you don’t already have it, install pip for Python 3 (not needed if you are using Python >= 3.4)
Install pyfastcopy: pip3 install pyfastcopy
Enjoy fast copy when using shutil.copyfile, shutil.copy, shutil.copy2 or shutil.copytree
From source
If you don’t already have it, install setuptools for Python 3
Clone this repository: git clone https://github.com/desbma/pyfastcopy
Install pyfastcopy: python3 setup.py install
Enjoy fast copy when using shutil.copyfile, shutil.copy, shutil.copy2 or shutil.copytree
License
Python Software Foundation License
1 maintainer
Release files for pyfastcopy 1.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pyfastcopy-1.0.3.tar.gz | 4.4 kB | Details |
Release files / pyfastcopy-1.0.3.tar.gz
| Download URL | pyfastcopy-1.0.3.tar.gz |
|---|---|
| Size | 4.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ed4620f1087a8949888973e315d3d59fbe9b8cc4ca5df553d76d2f21d2748999
|
|
BLAKE2b-256 checksum How to use checksums |
4380535d6b3de415e26d0a1cb774c6895dd07aa5986d2f8bde200393bd916790
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
