Skip to main content

Flex is a framework for building and executing computing pipelines

Project description

Muscle for data science

Data science projects get disorganized quickly. Every test involves a new script, each script requires a panoply of arguments and produces one or more data files. Keeping track of all this implied structure is a pain - what is this script for? What does it depend on? What created this data file? Which parameters updated this table in the database?

Enter flex - a utility that allows you to express and run all the computational tasks in a project. Crucially, it captures the specific parameters used for each task, the data files produced, and any dependencies that task has on other tasks. All this is captured in files called pipelines (which can even be connected to one another). Toss in some helpful comments, and you have executable documentation for your project.

It aims to achieve all this without introducing any overhead. You won’t have to write more code than you currently are doing, write or maintain any extra documentation, or use fancy data management solutions. Whatever you are already doing, flex is compatible with it.

Project details

Release history Release notifications | RSS feed

This version


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flexds-1.1.tar.gz (18.5 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page