Skip to main content

Use Git to record and deploy changes to your DevOps infrastructure

Project description

PyPI version

Introduction

Unfurl is a command line tool for deploying services and applications.

Describe your application architecture at a high-level using the OASIS TOSCA standard and Unfurl creates a plan adapted to your deployment environment (e.g. your cloud provider, Kubernetes, or self-hosted machines), using artifacts such as shell scripts, Terraform modules or Ansible playbooks.

After deployment, Unfurl records in git all the information needed for a reproducible deployment: the configuration, the artifacts and source repositories used in deployment, as well the state of the deployed resources. This enables intelligent updates as your configuration, dependencies, or environment changes.

The best way to manage your Unfurl project is to use Unfurl Cloud, our open-source platform for collaboratively developing cloud applications.

Why use Unfurl?

  • You are developing an application consisting of several interconnected services. Even a relatively simple web application has many dependencies that cut across its stack and development and deployment processes. Unfurl can help manage that complexity without complicated DevOps processes.

  • You have users that want to deploy your application on a variety of different cloud providers and self-hosted configurations. Support them by adding an adaptable blueprint to your source repository or publish it in a Cloud Map or on Unfurl Cloud, our open-source deployment platform.

  • You want to manage configuration and live resources just like code, using the same development processes -- like pull requests and CI pipelines -- without costly server infrastructure or having to rely on manual and proprietary admin UI.

How it works

  1. Create a project to manage your deployments.

Unfurl projects provide an easy way to track and organize all that goes into deploying an application, including configuration, secrets, and artifacts -- and track them in git. The goal is to provide isolation and reproducibility while maintaining human-scale, application-centric view of your resources.

  1. Create a cloud blueprint

Avoid tedious and error prone low-level configuration by creating a cloud blueprint that describes your cloud application's architecture at a high-level in Python or YAML using the OASIS TOSCA (Topology and Orchestration Specification for Cloud Applications) standard. Blueprints includes the resources (e.g. compute instances), the services (e.g. a database) and the artifacts (e.g. a Docker container image or a software package).

  1. Instantiate your blueprint

Unfurl instantiates your blueprint by building a deployment plan using implementations appropriate for the environment you are deploying into. For predefined types you can use our Unfurl Cloud Standard library for common cloud providers, kubernetes, and self-hosted instances. Or your can easily use your own implementations -- Unfurl integrates with tools like Terraform, Ansible, and Helm or you can encapsulate the scripts you already have.

  1. Deploy and Manage.

Run unfurl deploy to deploy the infrastructure. It will commit to git the latest configuration and a history of changes to your cloud accounts.

After the initial deployment, subsequent deployment plans take into account the current state of its resources. This way you can reduce errors by automatically reconfiguring as dependencies and environments change. You can also create custom operations or run ad-hoc commands in the ensemble's environment.

  1. Share and Collaborate

Unfurl stores everything in git so git is all you need to share projects and collaboratively manage deployments. Unfurl can use git lfs to provide locking during deployment and can automatically encrypts secrets. Or for ease of use and more advanced needs you can use unfurl cloud to host and deploy.

For complex applications and to enable open-source development and the integration of 3rd-party providers, Unfurl supports design patterns for integrating components and services that can be developed and deployed independently, such as encapsulation, composition, loosely-coupled components, dependency injection, semantic versioning, and component catalogs.

Our Vision

The ultimate goal of Unfurl is enable anyone to clone, fork, and deploy live cloud services as easily as cloning and building code from git. So that we can have:

  • Fully-functional and reproducible archives of web applications.
  • Location independence to decentralize cloud infrastructure.
  • Cooperatively build and run cloud services the same way we build open source software.

Features

No server, agentless

Simple, stand-alone CLI that can be used both in your local development environment or in an automated CI/CD pipeline.

Deploy infrastructure from simple, application-centric descriptions

  • Model your cloud infrastructure with the OASIS TOSCA (Topology and Orchestration Specification for Cloud Applications) standard either in YAML or Python.
  • Import reusable and adaptable components or build (and publish) your own.
  • Easily declare dependencies to enable incremental deployment.
  • Path-based query DSL to express dynamic relationships between resources and configurations.
  • Ansible-compatible Jinja2 templates provide an extensive, mature templating library.
  • Dynamic matching and substitution so models can adapt to different environments, services and cloud providers.

Integrates with the DevOps tools you are already using

Developer-friendly state management

  • All resource state is stored in git as human-readable and editable, application-centric resource state
  • You so can easily override or repair resource representations enabling interactive development.
  • Editor friendly config files:
    • Comments, order, and whitespace are preserved.
    • Syntactic macros for YAML provide generic facility for re-use and avoiding verbose boiler-plate.

Zero installation

  • Manage your local machine and bootstrap setup by deploying locally.
  • Downloads and installs specified software versions and code as deployment process.
  • Creates and manages isolated deployment environments.
  • Isolated environments can be either a Docker container or a Python virtualenv with asdf.
  • Clear separation of local and shared configuration to avoid unnecessary local configuration steps.

Flexible Secrets Management

  • Declare secrets and protect your repository.
  • Inline encryption or use external secret managers like HashiCorp Vault.
  • Automatic encryption of files in secrets folders.
  • Sensitive content redacted in output and logs

"Day Two" Operations

  • Check, discover, and repair commands
  • Define your own workflows for maintenance tasks like backup and restore.

Installation

unfurl is available on PyPI. You can install using pipx or pip:

pipx install unfurl

By default, Unfurl creates an Unfurl home project with a virtual Python environment for deployments so Unfurl only installs the minimal requirements needed to run the command line. If you want to deploy in the same environment, install it with the "full" option:

pipx install unfurl[full]

You can also install unfurl directly from this repository to get the latest code:

pipx install "git+https://github.com/onecommons/unfurl.git#egg=unfurl"

Alternatively, you can get the Unfurl container image from GitHub or Docker Hub and run Unfurl from the image:

docker run --rm -it -v $(pwd):/data -w /data onecommons/unfurl:stable unfurl ...

The stable tag matches the version published to PyPi; latest is the latest code from the repository.

Requirements

  • Linux or MacOS
  • Git
  • Python (3.8 - 3.13)

Optional: Docker or Podman

Shell autocomplete

Use the table below to activate shell autocompletion for the unfurl:

Shell Instructions
Bash Add this to ~/.bashrc:
eval "$(_UNFURL_COMPLETE=bash_source unfurl)"
Zsh Add this to ~/.zshrc:
eval "$(_UNFURL_COMPLETE=zsh_source unfurl)"
Fish Add this to ~/.config/fish/completions/unfurl.fish:
eval (env _UNFURL_COMPLETE=fish_source unfurl)

Development

To hack on Unfurl:

  1. Clone this repository with its submodules:

git clone --recurse-submodules https://github.com/onecommons/unfurl /path/to/unfurl/

  1. Install package dependencies and run Unfurl from source:

pip install -U -e /path/to/unfurl

  1. Build the Rust extension (this is optional for developing or running Unfurl):
pip install setuptools-rust>=1.7.0
python setup.py build_rust --debug --inplace

Requires Rust >= 1.70.

  1. To build documentation: Install tox (see below). Run tox -e docs.

  2. To build a distribution package run:

pip install build
python -m build /path/to/unfurl/

You can now install this package with pip or pipx, for example:

pipx install ./dist/unfurl-1.1.1.dev219-cp38-abi3-macosx_12_0_arm64.whl

Running unit tests

You can use tox to run the unit tests inside the supported python environments with the latest source installed. Install tox pip install tox==3.28.0 and then run tox in source root. To install the dependencies you may need header files installed by the following OS packages: python-dev, libcrypt-dev, openssl-dev. (Note: if installation of a dependency fails, reinvoke tox with -r to recreate the test environment.) If you use asdf to manage multiple versions of Python, also install tox-asdf: pip install tox-asdf.

Arguments after -- are passed to the test runner, e.g. to run an individual test: tox -- tests/test_runtime.py.

When running unit tests, tox it will try to build the Rust extension first. To skip building and running associated tests, set the UNFURL_TEST_SKIP_BUILD_RUST=1 environment variable.

Status and Caveats

Be mindful of these limitations:

  • Only clone and deploy trusted repositories and projects. The docker runtime is not configured to provide isolation so you should assume any project may contain executable code that can gain full access to your system.
  • Incremental updates are only partially implemented. You can incrementally update an ensemble by explicitly limiting jobs with the --force and --instance command line options.

Unfurl Cloud

The best way to manage your Unfurl project is to use Unfurl Cloud, our open-source platform for collaboratively developing cloud applications.

Get Started

Check out the rest of Unfurl's documentation here. Release notes can be found here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

unfurl-1.2.0.tar.gz (929.7 kB view details)

Uploaded Source

Built Distributions

unfurl-1.2.0-cp38-abi3-musllinux_1_1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.8+ musllinux: musl 1.1+ x86-64

unfurl-1.2.0-cp38-abi3-musllinux_1_1_aarch64.whl (1.6 MB view details)

Uploaded CPython 3.8+ musllinux: musl 1.1+ ARM64

unfurl-1.2.0-cp38-abi3-manylinux_2_28_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.8+ manylinux: glibc 2.28+ x86-64

unfurl-1.2.0-cp38-abi3-manylinux_2_28_aarch64.whl (1.5 MB view details)

Uploaded CPython 3.8+ manylinux: glibc 2.28+ ARM64

unfurl-1.2.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl (1.6 MB view details)

Uploaded CPython 3.8+ manylinux: glibc 2.17+ i686 manylinux: glibc 2.5+ i686

unfurl-1.2.0-cp38-abi3-macosx_10_12_universal2.whl (1.9 MB view details)

Uploaded CPython 3.8+ macOS 10.12+ universal2 (ARM64, x86-64)

File details

Details for the file unfurl-1.2.0.tar.gz.

File metadata

  • Download URL: unfurl-1.2.0.tar.gz
  • Upload date:
  • Size: 929.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for unfurl-1.2.0.tar.gz
Algorithm Hash digest
SHA256 f2fd744705b3a7a495a87ad827ca2ff492f79484c0aa17c2b4ba9d84be5e1e63
MD5 f9bf9af886bd94179281f09a3eb16e51
BLAKE2b-256 54b9c88c31864bbacc954f825e211dfbe379f2e54fcd3b9bb96e161ec2f1095e

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1e96a8d443571bf2e7e9f3113a35219427b58fa8b9ee378a4cf990a41595eb94
MD5 75599f099621fb6fa5417b6bd7c1c37e
BLAKE2b-256 7dfd1b8a573954490091e9aa1da1145e57e1606e6bb1778f64006b1115cecbed

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 7f485155eb1f540c0b2cf18a842644d6229b69862f2e7b58669c0e7cf375d262
MD5 441038c2a045375a41642257c6e121c0
BLAKE2b-256 4e00a73ff7f27858fa166b6db73153197b3b69cb9971cc047fa374042de4dbe9

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 80ca4ec991695338f28af1df64009268c21067ad4a2cef1533888883b016bd4d
MD5 5804f3df391e7369e7cd3b2adf56cc89
BLAKE2b-256 3777d92a866f0d52842edcd21d52878bc95aaccc0dd1f6c0ad530db55039e63b

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 336f1907e1f13cde2fbab7e93d2b7d2383a09a6feae5e22719abd7e6b740fa9e
MD5 228914d7807add8ecd6f7d9689999426
BLAKE2b-256 b681823b0587f6f9e2aaeddf2111c954f04c0f6cceb1e0023dafeda4bbf8471b

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 f1b4743b49842952c8bbb91c21bc2123f7130cf86062a972a6ba6b7bf6d24424
MD5 dc7d8f54306fd520e6bab7a1ad6095d0
BLAKE2b-256 316d1ce030ea7419892057bc707557a0d64b9290fda74b64afdc21084b7ef57d

See more details on using hashes here.

File details

Details for the file unfurl-1.2.0-cp38-abi3-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for unfurl-1.2.0-cp38-abi3-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 2b23c905c93f9f9e6a2a938c189933d3a07ea5875d6a52727a404c3cbff51fa1
MD5 f563774182ff39d48f44d314408a9347
BLAKE2b-256 29eceb053c4be662c0922d674f671751eae3a4ee0f5ca981f4dc609f28fcc5ba

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page