A small manager for versioned data
Project description
Flexible version control for files and folders.
Install
The simplest way is to get it from PyPi:
pip install bev
Or if you want to try the latest version from GitHub:
git clone https://github.com/neuro-ml/bev.git
cd bev
pip install -e .
# or let pip handle the cloning:
pip install git+https://github.com/neuro-ml/bev.git
Getting started
- Choose a folder for your repository and create a basic config (
.bev.yml
):
main:
storage: /path/to/storage/folder
meta:
hash: sha256
- Run
init
bev init
- Add files to bev
bev add /path/to/some/file.json .
bev add /path/to/some/folder/ .
bev add /path/to/some/image.png .
- ... and to git
git add file.json.hash folder.hash image.png.hash
git commit -m "added files"
- Access the files from python
import imageio
from bev import Repository
# `version` can be a commit hash or a git tag
repo = Repository('/path/to/repo', version='8a7fe6')
image = imageio.imread(repo.resolve('image.png'))
Advanced usage
Here are some tutorials that cover more advanced configuration, including multiple storage locations and machines:
- Create a repository - needed only at first time setup
- Adding files
- Accessing files
Why not DVC?
DVC is a great project, and we took inspiration from it while designing bev
.
However, out lab has several requirements that DVC
doesn't meet:
- Our data caches are spread across multiple HDDs - we need support for multiple cache locations
- We have multiple machines, and each of them has a different storage configuration: locations, number of HDDs, their volumes - we need a flexible way of choosing the right config depending on the machine
- Often we simultaneously conduct experiments on different versions of the same data - we need easy access to multiple version of the same data
- The need for
dvc checkout
aftergit checkout
is error-prone, because it can lead to situations when the data is not consistent with the current commit - we need a more constrained relation between data andgit
bev
supports all four out of the box!
However, if these requirements are not essential to your project, you may want to stick with DVC
- its community and
tests coverage is much larger.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bev-0.1.1.tar.gz
(19.0 kB
view hashes)