sw-aqueduct 2.0.2

A Swimlane content delivery system written in Python

Aqueduct

aq·ue·duct | \ˈä-kwə-ˌdəkt\

Definition of aqueduct

1. a conduit for water
2. a content delivery system for the Swimlane platform

(What's new?)

(Testing Steps)

Aqueduct is a Python package to migrate content from one (source) Swimlane instance to another (destination) instance in reliable and repeatable process.

Table of Contents

• Why?
• Features
  • Aqueduct Parameters
• Getting Started
• Usage Examples
• Installation
• Getting Help
• Contributing

Why?

Currently we only support transferring of content from one Swimlane instance to another that are the same semantic versioning number. You can bypass this but it is NOT recommended and there are limitations. Please see the available parameters for more information.

Many Swimlane customers build their own custom content (e.g. Use Cases) to fit their needs. We understand that every organization is different and because of this we have built aqueduct to facilitate the migration of this content from one instance to another. During the creation of aqueduct we have decided that we want to re-enforce best practices when using Swimlane. This is mainly in the form of NOT updating content in your production systems (e.g. destination systems). Changes should strictly be made within a development (source) instance and then migrated to your production instance once your testing is complete.

Features

By default aqueduct will NOT perform a live migration of data. You must specify dry_run=False in order to migrate content.

You can view all the available parameters for the main Aqueduct class here.

• By default, aqueduct will generate a .zip file containing source, altered, and destination JSONs when an error occurs.
• Providing the dump_content_path argument will dump ALL source, altered, and destination JSONs into a zip file.
• Aqueduct now reports back a list of homework items that must be completed manually on the destination.
  • You can access the homework list by using the Aqueduct._homework property.
• By using the mirror_app_fields_on_destination parameter, we will force update application fields.
• By using the update_default_reports parameter we will update Default reports.
• You can now add and update applets as a separate component.
• Collect related content based on one or more application names. Use the gather and collect methods.

Components

aqueduct can sync several different content components (listed below). Each of the listed component links below contain details on how content for that component is synced from one instance to another. Please read this for detailed information behind how aqueduct works.

Please be aware that the order of this list of components is forced whether you are syncing all components (default) or specific components from one Swimlane instance to another.

• keystore
• packages
• plugins
• assets
• workspaces
• applets
• applications
  • workflows
• tasks
• reports
• dashboards
• users
• groups
• roles

Getting Started

aqueduct is a Python-only package hosted on PyPi and works with Python 3.6 and greater.

pip install sw-aqueduct

Please see the Installation section for more information.

Usage Examples

As a Swimlane customer you can utilize aqueduct to migrate content from one instance to another. Additionally, you can utilize our Swimlane Aqueduct use case and plugin. To do so please reach out to customer support or your PS contact on how to acquire this SSP (Swimlane Solutions Package).

Before we look at the options available for configuring aqueduct, we must first create two SwimlaneInstance objects. One is considered the source instance and the other is the destination instance.

from aqueduct import SwimlaneInstance


sw_source = SwimlaneInstance(
    host="https://10.32.100.xxx",
    username="admin",
    password=""
)

sw_dest = SwimlaneInstance(
    host="https://10.32.100.xxx",
    username="admin",
    password=""
)

Once we have these instances configured, we can begin to configure aqueduct. You can view all the available parameters for the main Aqueduct class here.

By default aqueduct will NOT transfer your content. You must specify the dry_run=False parameter in order to actually transfer content.

from aqueduct import Aqueduct

# Below are the default parameters and values when instantiating a Aqueduct object.
aq = Aqueduct(
    source=sw_source,
    destination=sw_dest,
    dry_run=True,
    offline=False,
    update_reports=False,
    update_dashboards=False,
    continue_on_error=False,
    use_unsupported_version=False,
    force_unsupported_version=False,
    dump_content_path=None,
    update_default_reports=False,
    mirror_app_fields_on_destination=False,
)

Once we have configured out Aqueduct class instance we can now sync content from one instance to another. To do this we must call the sync method on the Aqueduct instance.

# this will sync all components listed above.
aq.sync()

By default we will sync all components but you can select which components you want to sync by passing in a list of components as an argument to the sync method.

# You can specify one or more of them as well.
aq.sync(components=['applications', 'plugins', 'workspaces', 'roles']

You can also include or exclude specific components themselves. Imagine you have an application and a plugin on your source instance that you do not want to transfer to a destination instance. You can specify this in an exclude (or include) parameter by passing in the following dictionary:

Please NOTE that if you specify a specific component item to exclude you must also provide that same component in the components parameter. If you do not, then it will run all components and only filter for that application name / plugin name as specified.

aq.sync(components=['applications', 'plugins'], exclude={'applications': ['My Source Application Name'], 'plugins': ['My Source Plugin Name']})

Installation

You can install aqueduct on OS X, Linux, or Windows. You can also install it directly from the source. To install, see the commands under the relevant operating system heading, below.

Prerequisites

The following libraries are required and installed by aqueduct:

pyyaml==6.0
fire==0.4.0
attrs==21.4.0
pydantic==1.9.1
swimlane==10.5.0
packaging>-21
requests>=2.27.1

macOS, Linux and Windows:

pip install sw-aqueduct

macOS using M1 processor

git clone https://github.com/swimlane/aqueduct.git
cd aqueduct

# Satisfy ModuleNotFoundError: No module named 'setuptools_rust'
brew install rust
pip3 install --upgrade pip
pip3 install setuptools_rust

# Back to our regularly scheduled programming . . .

python setup.py install

# or install via poetry
poetry install --build

Installing from source

git clone https://github.com/swimlane/aqueduct.git
cd aqueduct
python setup.py install
# or install via poetry
# poetry install

Getting Help

Please create an issue if you have questions or run into any issues.

Security Issues

Please read SECURITY.md for details on contacting us related to security issues.

Built With

• carcass - Python packaging template

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning.

Authors

• Josh Rickard - Initial work - MSAdministrator

See also the list of contributors who participated in this project.

License

This project is licensed under the MIT License - see the LICENSE file for details