Migrations to Astro Cloud
Project description
Apache Airflow Plugin meant to assist customers migrating Variables, Connections, and Environment Variables from a source Airflow environment to an Astronomer environment.
Initial Setup
Add the following line to your requirements.txt in your source environment:
astronomer-starship
Once your source environment installs astronomer-starship via the requirements.txt, you will see a new Astronomer menu. Hover over that menu and select the Migration Tool 🚀 option
After opening the plugin page, you will need to authenticate to Astronomer. To do this, please:
Click the Get Token button
If you are prompted to sign-in to cloud.astronomer.io please do so
Copy the access token that appears in the new tab
Paste the access token into the Authentication Token field
Click the Sign In button
After authenticating to Astronomer, you will need to select the deployment that you are sending metadata to. To do this, select a deployment from the Target Deployment dropdown and click the Select button
Migrating Airflow Connections
To migrate connections from your source Airflow meta-database:
Click on the Connections tab:
In the table displaying the connections that can be migrated, click the Migrate button for each connection that needs to be sent to the Target Deployment:
Once the Migrate button is clicked, the connection will be sent to the Target Deployment and will show as Migrated ✅ in the plugin UI:
Migrating Airflow Variables
To migrate variables from your source Airflow meta-database:
Click on the Variables tab:
In the table displaying the variables that can be migrated, click the Migrate button for each connection that needs to be sent to the Target Deployment
Once the Migrate button is clicked, the variable will be sent to the Target Deployment and will show as Migrated ✅ in the plugin UI:
Migrating Environment Variables
To migrate environment variables from your source Airflow:
Click on the Environment Variables tab:
In the table displaying the environment variables that can be migrated, ensure the checkbox is ticked for each environment variable that needs to be sent to the Target Deployment
Once all of the desired environment variable checkboxes have been selected, click the Migrate button in the table header
After clicking the Migrate button in the table header, each selected environment variable will be sent to the Target Deployment and the ticked checkbox will display Migrated ✅
Utilizing DAGs Cutover Tab
The DAGs Cutover Tab can be utilized to pause DAGs in the source environment and unpause DAGs in the target environment (as long as the DAG id in both the source and target environment match). To do so, please:
Click on the DAGs Cutover tab:
In the table displaying the DAGs present in both the source and target environments, click the Pause icon under Local
In the table displaying the DAGs present in both the source and target environments, click the Start icon under Remote
After completing this process, you will see the DAG is paused in the Local environment (so you will see a Start Icon) and is unpaused in the Remote environment (so you will see a Pause icon)
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for astronomer-starship-0.1.13.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8cd2fa7f01b45cd6ea6d01816b8cdf64acea6b3e3aa589fc49df3fb31b00b429 |
|
MD5 | 44400885e2df6d9380f5060fbd1cb0b9 |
|
BLAKE2b-256 | 906acb2a5bf3d9dfc171cbe08b40a0addb2cb8a76ed0e4c6e24cab42b3a86c7f |