Skip to main content

Common fixtures for inmanta LSM related modules

Project description

pytest-inmanta-lsm

A pytest plugin to test inmanta modules that use lsm, it is built on top of pytest-inmanta and pytest-inmanta-extensions

Installation

pip install pytest-inmanta-lsm

Context

This plugin is used to push code to a remote orchestrator and interact with it, via the LSM north-bound-api It requires an LSM enabled orchestrator, with no ssl or authentication enabled, in a default setup and ssh access to the orchestrator machine, with a user that has sudo permissions.

Usage

First case: using a remote orchestrator

This plugin is built around the remote_orchestrator fixture.

A typical testcase using this plugin looks as follows:

def test_full_cycle(project, remote_orchestrator):
    # get connection to remote_orchestrator
    client = remote_orchestrator.client

    # setup project
    project.compile(
        """
        import quickstart
        """
    )

    # sync project and export service entities
    remote_orchestrator.export_service_entities()

    # verify the service is in the catalog
    result = client.lsm_service_catalog_get_entity(remote_orchestrator.environment, SERVICE_NAME)
    assert result.code == 200

    # get a ManagedInstance object, to simplifies interacting with a specific service instance
    service_instance = remote_orchestrator.get_managed_instance(SERVICE_NAME)

    # create an instance and wait for it to be up
    service_instance.create(
        attributes={"router_ip": "10.1.9.17", "interface_name": "eth1", "address": "10.0.0.254/24", "vlan_id": 14},
        wait_for_state="up",
    )

    # make validation fail by creating a duplicate
    remote_orchestrator.get_managed_instance(SERVICE_NAME).create(
        attributes={"router_ip": "10.1.9.17", "interface_name": "eth1", "address": "10.0.0.254/24", "vlan_id": 14},
        wait_for_state="rejected",
    )

    service_instance.update(
        attribute_updates={"vlan_id": 42},
        wait_for_state="up",
    )

    # break it down
    service_instance.delete()

Second case: mocking the lsm api

This toolbox comes with one more fixture: lsm_project. This fixture allows to run compile using the lsm model locally. It has as advantage that:

  • You get a more fined grained control about what you want to see in your compile (choose the value of attributes, state, version, etc of your service).
  • If you only care about testing one specific case it is much faster than going through the full lifecycle on the remote orchestrator.
  • You don't need a running remote orchestrator, so you won't need to synchronize the full project anywhere.

A simple usage would be as follow:

def test_model(lsm_project: lsm_project.LsmProject) -> None:
    # Create a service object, you can modify it as you wish, depending on what you are trying to test
    service = inmanta_lsm.model.ServiceInstance(
        id=uuid.uuid4(),
        environment=lsm_project.environment,
        service_entity="vlan-assignment",
        version=1,
        config={},
        state="start",
        candidate_attributes={"router_ip": "10.1.9.17", "interface_name": "eth1", "address": "10.0.0.254/24", "vlan_id": 14},
        active_attributes=None,
        rollback_attributes=None,
        created_at=datetime.datetime.now(),
        last_updated=datetime.datetime.now(),
        callback=[],
        deleted=False,
        deployment_progress=None,
        service_identity_attribute_value=None,
    )

    # Add the service to the mocked server.  From now on it will be taken into account
    # for EACH compile
    lsm_project.add_service(service)

    # Run a compile, with central focus the service we just created
    lsm_project.compile("import quickstart", service_id=service.id)

Options and environment variables

The following options are available, each with a corresponding environment variable.

pytest-inmanta-lsm:
  --lsm-ca-cert
                        The path to the CA certificate file used to authenticate
                        the remote orchestrator. (overrides INMANTA_LSM_CA_CERT)
  --lsm-container-env   If set to true, expect the orchestrator to be running in
                        a container without systemd.  It then assumes that all
                        environment variables required to install the modules
                        are loaded into each ssh session automatically.
                        (overrides INMANTA_LSM_CONTAINER_ENV, defaults to False)
  --lsm-ctr             If set, the fixtures will deploy and orchestrator on the
                        host, using docker (overrides INMANTA_LSM_CONTAINER,
                        defaults to False)
  --lsm-ctr-cfg-file
                        A path to a config file that should be loaded inside the
                        container a server conf. (overrides
                        INMANTA_LSM_CONTAINER_CONFIG_FILE, defaults to
                        src/pytest_inmanta_lsm/resources/my-server-
                        conf.cfg)
  --lsm-ctr-compose-file
                        The path to a docker-compose file, that should be used
                        to setup an orchestrator (overrides
                        INMANTA_LSM_CONTAINER_COMPOSE_FILE, defaults to
                        src/pytest_inmanta_lsm/resources/docker-
                        compose.yml)
  --lsm-ctr-db-version
                        The version of postgresql to use for the db of the
                        orchestrator (overrides
                        INMANTA_LSM_CONTAINER_DB_VERSION, defaults to 10)
  --lsm-ctr-env-file
                        A path to an env file that should be loaded in the
                        container. (overrides INMANTA_LSM_CONTAINER_ENV_FILE,
                        defaults to
                        src/pytest_inmanta_lsm/resources/my-env-file)
  --lsm-ctr-image
                        The container image to use for the orchestrator
                        (overrides INMANTA_LSM_CONTAINER_IMAGE, defaults to
                        containers.inmanta.com/containers/service-
                        orchestrator:4)
  --lsm-ctr-jwe-file
                        A path to an entitlement file, required by the
                        orchestrator (overrides INMANTA_LSM_CONTAINER_JWE_FILE,
                        defaults to /etc/inmanta/license/com.inmanta.jwe)
  --lsm-ctr-license-file
                        A path to a license file, required by the orchestrator
                        (overrides INMANTA_LSM_CONTAINER_LICENSE_FILE, defaults
                        to /etc/inmanta/license/com.inmanta.license)
  --lsm-ctr-pub-key-file
                        A path to a public key that should be set in the
                        container (overrides INMANTA_LSM_CONTAINER_PUB_KEY_FILE,
                        defaults to $HOME/.ssh/id_rsa.pub)
  --lsm-environment
                        The environment to use on the remote server (is created
                        if it doesn't exist) (overrides INMANTA_LSM_ENVIRONMENT,
                        defaults to 719c7ad5-6657-444b-b536-a27174cb7498)
  --lsm-host=LSM_HOST   Remote orchestrator to use for the remote_inmanta
                        fixture (overrides INMANTA_LSM_HOST, defaults to
                        127.0.0.1)
  --lsm-no-clean        Don't cleanup the orchestrator after tests (for
                        debugging purposes) (overrides INMANTA_LSM_NO_CLEAN,
                        defaults to False)
  --lsm-srv-port
                        Port the orchestrator api is listening to (overrides
                        INMANTA_LSM_SRV_PORT, defaults to 8888)
  --lsm-ssh-port
                        Port to use to ssh to the remote orchestrator (overrides
                        INMANTA_LSM_SSH_PORT, defaults to 22)
  --lsm-ssh-user
                        Username to use to ssh to the remote orchestrator
                        (overrides INMANTA_LSM_SSH_USER, defaults to centos)
  --lsm-ssl             [True | False] Choose whether to use SSL/TLS or not when
                        connecting to the remote orchestrator. (overrides
                        INMANTA_LSM_SSL, defaults to False)
  --lsm-token
                        The token used to authenticate to the remote
                        orchestrator when authentication is enabled. (overrides
                        INMANTA_LSM_TOKEN)

Running tests

How the test suite is structured

The test suite consists of two parts:

  • The tests defined in tests/test_containerized_orchestrator.py file always run against a container started by the test suite itself.
  • All other tests run against the orchestrator specified by the options passed to the pytest command.

Pre-requisites

Testing (and using) pytest-inmanta-lsm requires:

  • an available orchestrator to test against
  • ssh access to this orchestrator

Steps

  1. install dependencies:
 pip install -r  requirements.dev.txt  -r  requirements.txt
  1. pass the config for pytest-inmanta-lsm via environment variables. e.g.
export INMANTA_LSM_HOST=<the orchestrator>
export INMANTA_LSM_USER=<user>
  1. set the repo for inmanta to pull LSM from
export INMANTA_MODULE_REPO=https://USER:LICENSE_TOKEN@modules.inmanta.com/git/inmanta-service-orchestrator/5/{}.git

Information on how to configure a Python package repository for V2 modules, instead of a Git URL, can be found here.

  1. run the tests
   pytest tests

Deploy a local orchestrator

It is possible to deploy an orchestrator locally and run the tests against it. The orchestrator will be deployed as a container, using docker. Here are the prerequisites in order to make it work:

  1. Have docker installed on your machine.

    $ docker version
    
  2. Have access to an orchestrator image (e.g. containers.inmanta.com/containers/service-orchestrator:4).

    $ export INMANTA_LSM_CONTAINER_IMAGE=containers.inmanta.com/containers/service-orchestrator:4
    $ docker pull $INMANTA_LSM_CONTAINER_IMAGE
    
  3. Have a license and an entitlement file for the orchestrator.

    $ ls /etc/inmanta/license/com.inmanta.*
    /etc/inmanta/license/com.inmanta.jwe  /etc/inmanta/license/com.inmanta.license
    $ export INMANTA_LSM_CONTAINER_LICENSE_FILE=/etc/inmanta/license/com.inmanta.license
    $ export INMANTA_LSM_CONTAINER_JWE_FILE=/etc/inmanta/license/com.inmanta.jwe
    
  4. Have a pair of private/public key to access the orchestrator.

    $ export PRIVATE_KEY=$HOME/.ssh/id_rsa
    $ if [ -f $PRIVATE_KEY ]; then echo "Private key already exists"; else ssh-keygen -t rsa -b 4096 -f $PRIVATE_KEY -N ''; fi
    $ export INMANTA_LSM_CONTAINER_PUB_KEY_FILE="${PRIVATE_KEY}.pub"
    $ if [ -f $INMANTA_LSM_CONTAINER_PUB_KEY_FILE ]; then echo "Public key already exists"; else ssh-keygen -y -f $PRIVATE_KEY > $INMANTA_LSM_CONTAINER_PUB_KEY_FILE; fi
    

If this is properly setup, you need to do set this option:

  --lsm-ctr        If set, the fixtures will deploy and orchestrator on the host, using docker (overrides INMANTA_LSM_CONTAINER, defaults to False)

Then any of the other option starting with lsm-ctr prefix to configure pytest-inmanta-lsm properly. You can specify:

  • The path to the license and entitlement files
  • The container image to use
  • The version of postgres to use
  • The public key to add in the orchestrator
  • Any env file that should be loaded by the orchestrator
  • A new docker-compose file to overwrite the one used by pytest-inmanta-lsm.
  • A new server config file

:warning: Some options have no effect when --lsm-ctr is set. This is the case of:

  • --lsm-host The host will be overwritten with the ip of the container
  • --lsm-srv-port The port will be overwritten with the port the server in the container is listening to
  • --lsm-ssh-port The port will be 22
  • --lsm-ssh-user The user will be inmanta
  • --lsm-container-env This is set to true automatically

:bulb: Some options change their behavior when --lsm-ctr is set. This is the case of:

  • --lsm-no-clean When set, the docker orchestrator won't be cleaned up when the tests are done. You will have to do it manually.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-inmanta-lsm-3.2.0.tar.gz (42.1 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page