Skip to main content

should make things more reproducible

Project description

DBIS Pipeline

This pipline can be used to run analyses in a structured way.

Usage

the user writes a minimal configuration file which contains only the following information:

  • "how do I get the data?", by providing a dataloader
  • "what to do with the data?", by providing a scikit pipeline
  • "what to do with the result?", by providing output handlers.

Please have a look at the examples for more information.

Requirements

pipenv

everything else should be installed by pipenv.

Installation

  1. Create a fresh directory.
  2. In that directory, call pipenv install dbispipeline This call will install a virtual environment as well as all dependencies.
  3. Write your configuration(s)
  4. call pipenv run python -m dbispipeline <yourconfigurationfile.py>

Enjoy!

Configuration

The framework look in multiple directories for its configuration files.

  • /usr/local/etc/dbispipeline.ini used for system wide default.
  • $HOME/.config/dbispipeline.ini used for user specific configurations.
  • ./dbispipeline.ini for project sepcifig configurations.

And example configuration file looks like this:

[database]

# url to your postgres database
host = your.personal.database

# your database user name
user = user

# port of your postgres database, default = 5432
# port = 5432

# password of your database user
password = <secure-password>

# database to use
database = pipelineresults

# table to be used
result_table = my_super_awesome_results

[project]
# this will be stored in the database
name = dbispipeline-test

[mail]
# email address to use as sender
sender = botname@yourserver.com

# recipient. This should probably be set on a home-directory-basis.
recipient = you@yourserver.com

# smtp server address to use
smtp_server = smtp.yourserver.com

# use smtp authentication, default = no
# authenticate = no

# username for smtp authentication, required if authenticate = yes
# username = foo

# password for smtp authentication, required if authenticate = yes
# password = bar

# port to use for smtp server connection, default = 465
# port = 465

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbispipeline-0.3.5.tar.gz (29.4 kB view hashes)

Uploaded Source

Built Distribution

dbispipeline-0.3.5-py3-none-any.whl (44.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page