Skip to main content

Mage is a tool for building and deploying data pipelines.

Project description

Mage AI

Give your data team magical powers.

Mage AI GitHub repo stars Mage AI Docker downloads Mage AI license Join the Mage AI community


Mage AI hero

Mage is a hybrid framework for transforming and integrating data. It combines the best of both worlds: the flexibility of notebooks with the rigor of modular code.


  • Extract and synchronize data from 3rd party sources.
  • Transform data with real-time and batch pipelines using Python, SQL, and R.
  • Load data into your data warehouse or data lake using our pre-built connectors.
  • Run, monitor, and orchestrate thousands of pipelines without losing sleep.

Plus hundreds of enterprise-class features, infrastructure innovations, and magical surprises.

Available in two spellbinding versions


Mage Pro For teams. Fully managed platform for integrating and transforming data. Mage OSS Self-hosted. System to build, run, and manage data pipelines.

Try out Mage Pro

It’s magic.

For documentation on getting started, how to develop, and how to deploy to production check out the live
Developer documentation portal.


🏃‍♀️ Install

The recommended way to install the latest version of Mage is through Docker with the following command:

docker pull mageai/mageai:latest

You can also install Mage using pip or conda, though this may cause dependency issues without the proper environment.

pip install mage-ai
conda install -c conda-forge mage-ai

Looking for help? The fastest way to get started is by checking out our documentation here.

Looking for quick examples? Open a demo project right in your browser or check out our guides.

🎮 Demo

Live demo

Build and run a data pipeline with our demo app.

WARNING

The live demo is public to everyone, please don’t save anything sensitive (e.g. passwords, secrets, etc).

Demo video (5 min)

Mage quick start demo

Click the image to play video


🔮 Features

🎶 Orchestration Schedule and manage data pipelines with observability.
📓 Notebook Interactive Python, SQL, & R editor for coding data pipelines.
🏗️ Data integrations Synchronize data from 3rd party sources to your internal destinations.
🚰 Streaming pipelines Ingest and transform real-time data.
dbt Build, run, and manage your dbt models with Mage.

A sample data pipeline defined across 3 files ➝


  1. Load data ➝
    @data_loader
    def load_csv_from_file() -> pl.DataFrame:
        return pl.read_csv('default_repo/titanic.csv')
    
  2. Transform data ➝
    @transformer
    def select_columns_from_df(df: pl.DataFrame, *args) -> pl.DataFrame:
        return df[['Age', 'Fare', 'Survived']]
    
  3. Export data ➝
    @data_exporter
    def export_titanic_data_to_disk(df: pl.DataFrame) -> None:
        df.to_csv('default_repo/titanic_transformed.csv')
    

Water mage casting spell

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mage_ai-0.9.76.tar.gz (38.1 MB view details)

Uploaded Source

Built Distribution

mage_ai-0.9.76-py3-none-any.whl (39.9 MB view details)

Uploaded Python 3

File details

Details for the file mage_ai-0.9.76.tar.gz.

File metadata

  • Download URL: mage_ai-0.9.76.tar.gz
  • Upload date:
  • Size: 38.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for mage_ai-0.9.76.tar.gz
Algorithm Hash digest
SHA256 ca72856e7111477f078226e4cbcb3c6f59ce49202afa09d7cd0c79aeedd6855a
MD5 a8e0683bd2047e2830a0aedfe7d5ac11
BLAKE2b-256 85d1ab4f74a2a365609e2110a95cfa6d8d9192e57fb528073a5ff272b46324ad

See more details on using hashes here.

File details

Details for the file mage_ai-0.9.76-py3-none-any.whl.

File metadata

  • Download URL: mage_ai-0.9.76-py3-none-any.whl
  • Upload date:
  • Size: 39.9 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for mage_ai-0.9.76-py3-none-any.whl
Algorithm Hash digest
SHA256 96ace553c885f3113ed496c90b6309f607bec767a317a2425651b1e06330e75c
MD5 c3b1477b75e6e2ec986c6568af660cf3
BLAKE2b-256 7efbb1632e6aec3a93d92f94e1e0929d417ae4d5ed946efdb472172490abfe69

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page