Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

should make things more reproducible

Project description

DBIS Pipeline

This pipline can be used to run analyses in a structured way.

Usage

the user writes a minimal configuration file which contains only the following information:

  • "how do I get the data?", by providing a dataloader
  • "what to do with the data?", by providing a scikit pipeline
  • "what to do with the result?", by providing output handlers.

Please have a look at the examples for more information.

Requirements

pipenv

everything else should be installed by pipenv.

Installation

  1. Create a fresh directory.
  2. In that directory, call pipenv install dbispipeline This call will install a virtual environment as well as all dependencies.
  3. Write your configuration(s)
  4. call pipenv run python -m dbispipeline <yourconfigurationfile.py>

Enjoy!

Configuration

The framework look in multiple directories for its configuration files.

  • /usr/local/etc/dbispipeline.ini used for system wide default.
  • $HOME/.config/dbispipeline.ini used for user specific configurations.
  • ./dbispipeline.ini for project sepcifig configurations.

And example configuration file looks like this:

[database]

# url to your postgres database
host = your.personal.database

# your database user name
user = user

# port of your postgres database, default = 5432
# port = 5432

# password of your database user
password = <secure-password>

# database to use
database = pipelineresults

# table to be used
result_table = my_super_awesome_results

[project]
# this will be stored in the database
name = dbispipeline-test

[mail]
# email address to use as sender
sender = botname@yourserver.com

# recipient. This should probably be set on a home-directory-basis.
recipient = you@yourserver.com

# smtp server address to use
smtp_server = smtp.yourserver.com

# use smtp authentication, default = no
# authenticate = no

# username for smtp authentication, required if authenticate = yes
# username = foo

# password for smtp authentication, required if authenticate = yes
# password = bar

# port to use for smtp server connection, default = 465
# port = 465

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dbispipeline, version 0.2.1
Filename, size File type Python version Upload date Hashes
Filename, size dbispipeline-0.2.1-py3-none-any.whl (41.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size dbispipeline-0.2.1.tar.gz (27.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page