Methods to help track the scripts and datafiles in a project.
Project description
Datatracker
Datatracker is a basic logging Python package that keeps track of files and code within a Project. Each script is logged as an entry and input and output datafiles are recorded in order. Datatracker is able to manage versioning of both files and scripts, and is able to identify the most up-to-date version.
At the moment, this Python package is still in alpha, and I may include changes to both UI and file format that may be breaking.
Installation
To install, run the following command:
pip install git+ssh://git@github.com/TarjinderSingh/datatracker
Usage
New entries
from datatracker import *
tr = Tracker()
os.environ['VERSION'] = '0.1.0'
entry = Entry(tag='filter-common-variants',
description='Filtering common variants in new GWAS data set.',
category='Processing',
module='Variant QC')
infile = entry.add(
InputFile(tag='raw-plink-file',
path='gs://bucket/raw-plink-file.bed',
description='Raw PLINK file.'))
outfile = entry.add(
OutputFile(tag='filt-plink-file',
path='gs://bucket/raw-plink-file.bed',
description='Filtered PLINK file.'))
tr.save(entry)
View existing entries
from datatracker import *
tr = Tracker()
tr.table
Use existing entries for pipeline
infile = entry.add(InputFile(entry_tag='filter-common-variants', tag='raw-plink-file', database=tr))
Filter and remove
# filter to entry
tr.filter(tr.entry.tag_version == 'import-array_0.1.6')
# remove entry
tr.remove(tr.entry.tag_version == 'import-array_0.1.6')
Pandas and Excel
df = tr.explode()
df = tr.explode('filt-plink-file')
df = tr.to_pandas()
df = tr.table
df.to_excel('spreadsheet.xlsx')
Data artifacts
infile = entry.add(InputFile(path='gs://checkpoint-cache/tmp/1.bed'))
License
MIT License (see repository)
Maintainer
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
datatracker-0.2.0.tar.gz
(22.9 kB
view hashes)
Built Distribution
Close
Hashes for datatracker-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec12ba16f5f83f30252aae4d36cd9f5c3a491396a462c23bb5eb0a4bcef940a0 |
|
MD5 | 7f3040d33c3eef30f84b8f064d466672 |
|
BLAKE2b-256 | c169ebc1d63f9b36c53a9c70b930371352e99de770ccbaee9509de1a8aa534ed |