A commandline tool for analysis of big biological data sets for distributed HPC clusters.
About
PyBDA is a Python library and command line tool for big data analytics and machine learning scaling to tera byte sized data sets.
In order to make PyBDA scale to big data sets, we use Apache Spark’s DataFrame API which, if developed against, automatically distributes data to the nodes of a high-performance cluster and does the computation of expensive machine learning tasks in parallel. For scheduling, PyBDA uses Snakemake to automatically execute pipelines of jobs. In particular, PyBDA will first build a DAG of methods/jobs you want to execute in succession (e.g. dimensionality reduction into clustering) and then compute every method by traversing the DAG. In the case of a successful computation of a job, PyBDA will write results and plots, and create statistics. If one of the jobs fails PyBDA will report where and which method failed (owing to Snakemake’s scheduling) such that the same pipeline can effortlessly be continued from where it failed the last time.
Documentation
Check out the documentation here. The documentation will walk you though
the installation process,
setting up Apache Spark,
using pybda.
1 maintainer
Metadata
1 maintainer
Release files for pybda 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| pybda-0.1.0.tar.gz | 55.0 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pybda-0.1.0-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Total release size: 182.2 kB
Release files / pybda-0.1.0.tar.gz
| Download URL | pybda-0.1.0.tar.gz |
|---|---|
| Size | 55.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5549d969b88f31201f4e03e8cb5864f0f8c80442369b23241e72973956f09880
|
|
BLAKE2b-256 checksum How to use checksums |
80189ca71b566948e42e938a0d5ddf5d5cefc27c262ee7e94f161794db2d1eaa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7
|
Release files / pybda-0.1.0-py2.py3-none-any.whl
| Download URL | pybda-0.1.0-py2.py3-none-any.whl |
|---|---|
| Size | 127.2 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
de58bf9bbdbf452af77079a37385de7410a4b66f077c4df5f36e0190907419b9
|
|
BLAKE2b-256 checksum How to use checksums |
9b28c1d9647212d0cc0cc5272077d5456a6d95b52c8b159c18b5c633902cc285
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7
|
Security for pybda 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
