The easiest way to create data pipelines in Python.
Project description
Prism is currently in alpha.
:wave: Welcome to Prism!
Prism is the easiest way to create data pipelines in Python.
Introduction
Data projects often require multiple steps that need to be executed in sequence (think extract-transform-load, data wrangling, etc.). With Prism, users can break down their project into modular tasks, manage dependencies, and execute complex computations in sequence.
Here are some of Prism's main features:
- Real-time dependency declaration: With Prism, analysts can declare dependencies using a simple function call. No need to explicitly keep track of the pipeline order — at runtime, Prism automatically parses the function calls and builds the dependency graph.
- Intuitive logging: Prism automatically logs events for parsing the configuration files, compiling the tasks and creating the DAG, and executing the tasks. No configuration is required.
- Flexible CLI: Users can instantiate, compile, and run projects using a simple, but powerful command-line interface.
- “Batteries included”: Prism comes with all the essentials needed to get up and running quickly. Users can create and run their first DAG in less than 2 minutes.
- Integrations: Prism integrates with several tools that are popular in the data community, including Snowflake, Google BigQuery, Redshift, PySpark, and dbt. We're adding more integrations every day, so let us know what you'd like to see!
Getting Started
Prism can be installed via pip
. Prism requires Python >= 3.7.
pip install --upgrade pip
pip install prism-ds
Prism also supports several adapters! They can be installed with the following commands:
Adapter | Command |
---|---|
dbt | pip install "prism-ds[dbt]" |
Google BigQuery | pip install "prism-ds[bigquery]" |
Postgres | pip install "prism-ds[postgres]" |
PySpark | pip install "prism-ds[pyspark]" |
Redshift | pip install "prism-ds[redshift]" |
Snowflake | pip install "prism-ds[snowflake]" |
Trino | pip install "prism-ds[trino]" |
To get started with Prism projects, check out our documentation. Some sections of interest include:
- :key: Fundamentals
- :seedling: CLI
- :electric_plug: Integrations
- :bulb: Use Cases
In addition, check out some example projects.
Product Roadmap
We're always looking to improve our product. Here's what we're working on at the moment:
- Triggers: trigger function calls on the success and/or failure of your project
- Additional adapters: Celery, Dask, Docker, and more!
- Cloud deployment: managed orchestration platform to deploy Prism projects in the cloud
Let us know if you'd like to see another feature!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for prism_ds-0.1.7rc1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cda60661e99813562ca3cc574bb3e013762699f97ded6643d1ca8859f11f8a2c |
|
MD5 | 6f382afffdfa9fcb783065f4d731262e |
|
BLAKE2b-256 | c38d82fd6418d6dbade2328c91ac8dbaff84a6cbe035c319c1ac6b17adf26ab9 |