Filestore
The filestore is an easy way to to process files with multiple processes without needing locks.
Initialize a FileStore
Create a filestore in a temporary area:
>>> import tempfile, os >>> temp_dir = tempfile.mkdtemp() >>> store_dir = os.path.join(temp_dir, 'store1') >>> os.mkdir(store_dir) >>> from gocept.filestore import FileStore >>> filestore = FileStore(store_dir) >>> filestore <gocept.filestore.filestore.FileStore object at 0x...>
So far nothing has happend:
>>> import os >>> os.listdir(store_dir) []
Before using the store we need to prepare it:
>>> filestore.prepare()
Prepare has created the tmp/new/cur directory structure:
>>> sorted(os.listdir(store_dir)) ['cur', 'new', 'tmp']
Calling prepare again does nothing:
>>> filestore.prepare() >>> sorted(os.listdir(store_dir)) ['cur', 'new', 'tmp']
If the store_dir is removed, it is created again by calling prepare.
>>> import shutil >>> shutil.rmtree(store_dir) >>> os.listdir(temp_dir) [] >>> filestore.prepare() >>> os.listdir(temp_dir) ['store1'] >>> sorted(os.listdir(store_dir)) ['cur', 'new', 'tmp']
Use a FileStore
Adding files to the store works with the create method:
>>> f = filestore.create('a-file')
Files are created in the ‘tmp’ area with the ‘w’ mode (if not specified):
>>> f.name '.../tmp/a-file' >>> f.mode 'w'
We find the file in the tmp area. Note that filestore.list lists files with their full path names, so we could feed the name directly to file/open:
>>> filestore.list('tmp')
['.../tmp/a-file']
We got a plain file back, so write to it:
>>> _ = f.write('Die Ente bleibt draussen!')
>>> f.close()
We have finished writing our file, so we can move it to the new space for other applications to pick it up:
>>> filestore.move('a-file', 'tmp', 'new')
>>> filestore.list('tmp')
[]
>>> filestore.list('new')
['.../new/a-file']
The move operation uses os.move which is supposed to be atomic. When another processes “sees” the file it can directly work with it and move it to ‘cur’:
>>> filestore.move('a-file', 'new', 'cur')
>>> filestore.list('new')
[]
>>> filestore.list('cur')
['.../cur/a-file']
Files can be copied, too:
>>> filestore.copy('a-file', 'cur', 'tmp')
>>> filestore.list('cur')
['.../cur/a-file']
>>> filestore.list('tmp')
['.../tmp/a-file']
Finally, files can be removed:
>>> filestore.remove('a-file', 'cur')
>>> filestore.list('cur')
[]
Cleanup
Remove the temporary directory after testing:
>>> import shutil >>> shutil.rmtree(store_dir)
Changes
1.0 (2023-07-14)
Drop support for Python 2.7, 3.5, 3.6.
0.5 (2023-03-16)
Add support for Python 3,9, 3.10, 3.11.
Use GitHub actions as CI.
0.4 (2019-11-29)
Migrate repository to Bitbucket.
Migrate repository to GitHub.
Made Python 3 compatible (tested with Python 2.7, 3.7 and 3.8).
Replace bootstrap/buildout with tox.
Increase test coverage to 100%.
0.3 (2009-10-08)
Added copy() method.
0.2 (2007-08-30)
Initial public release.
Release files for gocept.filestore 1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| gocept.filestore-1.0.tar.gz | 6.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| gocept.filestore-1.0-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 13.7 kB
Release files / gocept.filestore-1.0.tar.gz
| Download URL | gocept.filestore-1.0.tar.gz |
|---|---|
| Size | 6.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3fbec68b20cf2b2df54d046aafb6a382a83a772e1b5c59b459c44ec4dd843786
|
|
BLAKE2b-256 checksum How to use checksums |
e5a798c1abc3c87a7a7a3aade652b7fea069c6a49429f14e087451bdcf0e7f52
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.4
|
Release files / gocept.filestore-1.0-py2.py3-none-any.whl
| Download URL | gocept.filestore-1.0-py2.py3-none-any.whl |
|---|---|
| Size | 7.4 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
f2488366f26e8c32e7c53fcb09e8d22c1918c4bcbbeba6a9afa1324c471bbcc6
|
|
BLAKE2b-256 checksum How to use checksums |
8f25069ec6906b96827a4ccb9ed6af458964d7320c8e331a5d5de2da57a4451b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.4
|