Mage is a notebook for building and deploying data pipelines.
Project description
Intro
Mage is an open-source notebook for building and deploying data pipelines.
Table of contents
🏃♀️ Quick start
You can install Mage using Docker or pip:
Using Docker
1. Clone repository
git clone https://github.com/mage-ai/mage-ai.git && cd mage-ai
2. Create new project
./scripts/init.sh [project_name]
3. Launch editor
./scripts/start.sh [project_name]
Open http://localhost:6789 in your browser and build a pipeline.
4. Run pipeline after building it in the tool
./scripts/run.sh [project_name] [pipeline]
Using pip
1. Install Mage
pip install mage-ai
You may need to install development libraries for MIT Kerberos to use some Mage features. On Ubuntu, this can be installed as:
apt install libkrb5-dev
2. Create new project
mage init [project_name]
3. Launch editor
mage start [project_name]
Open http://localhost:6789 in your browser and build a pipeline.
4. Run pipeline after building it in the tool
mage run [project_name] [pipeline]
👩🏫 Tutorials
- Train model on Titanic dataset
- Load data from API, transform it, and export it to PostgreSQL
- [Coming soon] How to visualize your data with charts
🔮 Features
1. Data centric editor
An interactive coding experience designed for preparing data to train ML models.
Visualize the impact of your code every time you load, clean, and transform data.
2. Production ready code
No more writing throw away code or trying to turn notebooks into scripts.
Each block (aka cell) in this editor is a modular file that can be tested, reused, and chained together to create an executable data pipeline locally or in any environment.
Read more about blocks and how they work.
Run your data pipeline end-to-end using the command line function: $ mage run [project] [pipeline]
You can run your pipeline in production environments with the orchestration tools
3. Extensible
Easily add new functionality directly in the source code or through plug-ins (coming soon).
Adding new API endpoints (Tornado), transformations (Python, PySpark, SQL), and charts (using React) is easy to do (tutorial coming soon).
New features and changelog
Check out what’s new here.
🙋♀️ Contributing
We welcome all contributions to Mage; from small UI enhancements to brand new cleaning actions. We love seeing community members level up and give people power-ups!
Check out the 🎁 contributing guide to get started by setting up your development environment and exploring the code base.
Got questions? Live chat with us in
Anything you contribute, the Mage team and community will maintain. We’re in it together!
🧙 Community
We love the community of Magers (/ˈmājər/);
a group of mages who help each other realize their full potential!
To live chat with the Mage team and community,
please join the free Mage
For real-time news and fun memes, check out the Mage
To report bugs or add your awesome code for others to enjoy, visit GitHub.
🪪 License
See the LICENSE file for licensing information.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
