Skip to main content

Tools to help developers build services that implement the Action Provider specification.

Project description

Action Provider Tools Introduction

This is an experimental and unsupported toolkit to help developers build Action Providers for use in Globus Automate including for invocation via Globus Flows.

As this is experimental, no support is implied or provided for any sort of use of this package. It is published for ease of distribution among those planning to use it for its intended, experimental, purpose.

This README is intended to introduce the concepts, requirements and overview of this toolkit to be used when implementing an Action for use with Globus Automate. The three main sections are 1) an introduction to the interface required of all Actions, 2) A description of how to prepare your service to use Globus Auth, and 3) A description of the content of this toolkit and how it can be used in a Python implementation. This overview does not provide any guidance on how to host or operate the Action so that it is accessible via the internet.

Actions Overview

The fundamental purpose of the Globus Automate platform is to tie together multiple operations or units of work into a coordinated orchestration. We refer to each of these operations as an Action. In particular, the Flows service provides a means of coordinating multiple actions across potentially long periods of time to perform some aggregate function larger than any single Action provides. The Triggers service ties Events, or occurrences within a managed environment, to Actions such that each occurrence of the Event automatically invokes the Action associated with it.

In both the Flows and the Triggers cases, the Actions require a uniform interface for invocation, monitoring and possibly termination so that new Actions may be introduced without requiring customization or re-implementation of the invoking services. We refer to the service endpoints which can be invoked in this manner as Action Providers and the uniform interface for interacting with the Action Providers as the Action Provider Interface. We provide here an overview of the Action Provider Interface as a guide for use when implementing an Action Provider.

The Action Provider Interface

The Action Provider Interface is a RESTful model for starting, monitoring, canceling and removing state associated with the invocation of an Action. Following the REST resource life-cycle pattern, each Action invocation returns an identifier representing the invocation (an Action Instance). This identifier is used to monitor the progress of the Action Instance via further REST calls until its completion, or it may be used to request cancellation of the instance.

Because the interface is intended to support arbitrary Action types, we recognize that some Action instances may be long-running (asynchronous) such as the execution of a computational job. Other Actions may be short-running (synchronous), able to return their final result directly in response to their invocation request as is the case in typical RESTful models. The Action Life-cycle described below specifically supports these execution modes as well as handling failures and Actions which may be, temporarily, unable to make progress.

Action Life-cycle

The Life-cycle of an Action defines the set of states that the Action may be in, and how it can transition among the states. The states are defined as follows:

  • ACTIVE: The Action is executing and is making progress toward completion

  • INACTIVE: The Action is paused in its execution and is not making progress toward completion. Out-of-band (i.e. not via the Action Provider Interface) measures may be required to allow the Action to proceed.

  • SUCCEEDED: The Action reached a completion state which was considered “normal” or not due to failure or other unrecoverable error.

  • FAILED: The Action is in a completion state which is “not normal” such as due to an error condition which is not considered recoverable in any manner.

  • “Released”: The Action Provider has removed the record of the existence of the Action. Further attempts to interact with the Action will be errors as if the Action had never existed. All resources associated with the Action may have been deleted or removed. This is not a true state in the sense that the state can be observed, but ultimately all Actions will be released and unavailable for further operations. Any subsequent references to the Action, e.g. via the API methods described below, will behave as if the Action never existed.

Upon initial creation of an Action (see operations below), the Action may be in any of the first four states. If it is in an ACTIVE or INACTIVE state, the Action is considered “asynchronous” and further queries to get the state of the Action may return updated information. If the Action is in a the SUCCEEDED or FAILED states, the Action is synchronous, all information about the Action is returned on the creation operation and no changes to the state are possible.

An asynchronous Action may change state between ACTIVE and INACTIVE during its life time, and may update further details about its progress while in either of these states. When a completed state of SUCCEEDED or FAILED is reached, the Action state cannot be updated further. The Action Provider is, however, required to maintain this final state for some period of time so that the client of the Action may retrieve the completion state. Upon completion, the client may request that the Action be “released” or the Action Provider may do so on its own after the required time-out occurs. To save server resources, it is preferred that the client release the Action when it has reliably retrieved and processed the final state.

Action Provider Interface and Document Types

The primary purpose of the Action Provider Interface is to securely support and report Actions progressing through the life-cycle described above. The document types supporting this are the initial Action invocation Action Request document, and the Action Status document which contains the life-cycle status described above along with additional detailed status information specific to the type of Action being executed.

Starting an Action: The Action Request Document and the POST /run Method

Starting an Action is performed by making a REST POST request to the path /run containing an Action Request document. The request document contains the following fields:

  • request_id: A client-generated Identifier for this request. A user may re-invoke the /run method with the same request_id any number of times, but the Action must only be initiated once. In this way, the user may re-issue the request in case it cannot be determined if a request was successfully initiated for example due to network failure.

  • body: An Action Provider-specific object which provides the input for the Action to be performed. The body must conform to the input specification for the Action Provider being invoked, and thus the client must understand the requirements of the Action Provider when providing the value of the body. Thus, the Action Provider must provide documentation on the format for the body property.

  • manage_by and monitor_by: Each of these is a list of principal values in URN format, and they allow the user invoking the Action to delegate some capability over the Action to other principals. manage_by defines the principals who are allowed to attempt to change the execution of the Action (see operations /cancel and /release below) while it is running. monitor_by defines principals which are allowed to see the state of the Action before its state has been destroyed in a release operation. In both cases, the Globus Auth identity associated with the /run operation is implicitly part of both the manage_by and monitor_by sets. That is, the invoking user need not include their own identity into these lists.

Any request to the /run method which contains an Action Request which adheres to the input schema will return an Action Status document as described in the next section.

Monitoring and Completing an Action: The Action Status Document and Management Methods

All information about an Action is contained in the Action Status document which is returned on almost all operations related to an Action (the exception is the log operation which is optional and is described briefly below). Notable fields of the Action Status document include:

  • action_id: The unique identifier for this particular action. The action_id may be any string, and it should be treated as an opaque value (that is, having no semantic or implied meaning) by the client. The client will first learn of an Action’s action_id in the Action Status returned by the /run method.

  • status and display_status: These provide the description of the state of the Action. status is the specific life-cycle value as described above. display_status is an optional field the Action Provider may supply which gives a short text description of the status using language which is specific to the Action.

  • details: The Action Provider-specific state, particularly the completion state, of the Action are returned in the details field. In the completion states, the details can be considered the “result” or the “return value” of the Action. It is the successful return value for a SUCCEEDED status, and it is the error result for the FAILED status. The exact content in details is always specific to the Action Provider, so must be documented by the Action Provider to describe its interpretation to clients.

  • monitor_by and manage_by: Same as in the Action Request.

  • start_time and completion_time: Represent the time the Action was first received by the /run operation and the time the Action Provider determined that the Action reached a completed state (SUCCEEDED or FAILED) respectively. Action Providers are not required to continuously monitor the progress of Actions, so the completion_time noted may be different than the executed Action’s actual completion time. These values may be the same in the case of a synchronous operation, but completion_time must never be before start_time.

  • release_after: As stated above, Action state is automatically removed from the Action Provider after some time interval once it reaches a completion state. The release_after is a time duration, in seconds, which states how long after completion the Action will automatically be released. A typical value would be 30-days, but Action Providers may define their own policy which is to be exposed in the Action Status.

In addition to the /run method described above, the Action Status is the “universal” return value from operations on an Action. We describe the operations on Actions next. Each uses the action_id as part of the URL path much like other RESTful resources do with their ids, and none of them require any input body.

  • GET /<action_id>/status: This is a read-only operation for retrieving the most recent state of the Action. It is commonly used to poll an Action’s state while awaiting it entering a completion state. Use of this API call requires that the user authenticate with a principal value which is in the monitor_by list established when the Action was started.

  • POST /<action_id>/cancel: Cancellation provides an advisory or hint to the Action Provider that the user does not want the Action to continue execution. The Action Provider is not required to ensure immediate completion or that the cancel operation truly causes the Action to terminate in any manner other than it would have without the cancel request. Thus, the Action Status returned from the cancel operation may contain a non-completion state. If the Action is already in a completed state, the Action Provider may treat the request much as a /status request to simply return the current status. Use of this API call requires that the user authenticates with a principal value which is in the manage_by list established when the Action was started.

  • POST /<action_id>/release: As described in the section on life-cycle, the very last step of the life-cycle is for the Action state to be removed from the Action Provider. A user can specify that it has retrieved the final state or is no longer interested in the state using the /release operation which returns the final state. If the Action is not already in a completion state, /release will return an error as this operation does not attempt to stop execution (that is what /cancel does). The Action Status document returned from /release will be the last record of the Action present at the Action Provider. After the call to /release the action_id is no longer valid, and use in any other calls will return an error, most likely an HTTP status 404 indicating the Action was not found.

Detailed Execution History: logging

Some Actions, particularly those that are long running, may have associated with them a list or log of activities or sub-events which occurred during the Action’s life. This detailed log is typically larger, more complex, or more fine-grain than the snapshot of the status returned by the /status method. Not all Action Providers or Actions are suitable for logging, so support is considered optional and will be advertised by the Action Provider in its description (see below). The request to retrieve the log takes the form GET /<action_id>/log?<filters,pagination>. The filters and pagination query parameters are used to limit (e.g. based on start time) which log records to retrieve and the pagination parameter is used to scroll through a long set of log records across multiple requests. Each record in the log contains the following properties:

  • time: A timestamp representing the time this log record occurred.

  • code: A short Action Provider-specific description of the type of the log record.

  • description: A textual description of the purpose, cause, or information on the log record.

  • details (optional): An object providing additional and structured Action Provider-specific representation of the log record.

Action Provider Introspection (GET /)

The Automate platform is intended to help users both find and make use of the variety of Action Providers which may be available on the network. The primary means of accomplishing this assistance is by making Action Providers, the services which implement the Action Provider Interface, self-describing via an Introspection interface. Accessing the introspection method is performed simply via a GET /. That is, the HTTP GET method on the Base URL. The returned JSON document contains the following fields:

  • api_version: A version string defining the version of the Action Provider Interface supported by the Action Provider. The version described in this document and currently the only version available will have value "1.0".

  • title, subtitle, description, keywords: Each of these provide human-readable text which helps a user discover the purpose of the Action Provider.

  • visible_to and runnable_by: Access to the action provider is limited by and published through these properties. Each contains a list of principal values in URN format. visible_to controls who can retrieve the information via introspection (this operation) and may contain the string "public" indicating that all users, even those who present no credentials, may access the information. The runnable_by property enumerates who can use the /run method to start an Action at this provider. It allows the string "all_authenticated_users" indicating that any user who presents valid credentials via a Bearer token may start an Action at the provider.

  • synchronous and log_supported: These are boolean values which simply describe capabilities and modes for the Action Provider. If synchronous is true, a user calling /run can assume that the returned status will always be in a completed (SUCCEEDED or FAILED) state and there will never be a need to poll using the /status method (use of /release is still permitted and encouraged to remove the status from the Action Provider). As indicated in the discussion of the /log method, support for it is optional, and the log_supported flag provides an indication to users whether they can make use of /log for fine grained monitoring of an Action.

  • input_schema: The input_schema value provides a complete schema description for the body property of the Action Request understood by this Action Provider. The schema is provided in JSON Schema format.

Setting Up an Action Provider in Globus Auth

The Action Provider Interface makes use of and is bound closely with authentication via the Globus Auth system. To authenticate RESTful requests using Globus Auth, a service must register as a “resource server”. This is a multi-step process involving use of both the Globus Auth developer portal, and the Globus Auth API for configuring various access control states. To help with this process, we provide a step-by-step guide to using Globus Auth for this purpose:

  1. Register a new App on https://developers.globus.org using a browser. After insuring that you are logged in to the developer portal in a browser at this URL, perform the following steps:

    • Click Add another project

      • Provide a name, contact email and select which of your own Globus Auth linked identities are permitted to administer the project. You will be required to login with this identity in future interactions with the Globus Developer Portal to manipulate the resource server.

    • Find your new, empty project, and select Add drop down and “new app”

      • Provide a name for the specific app within the project. This will be a common name displayed to users when they make use of the Action Provider.

      • When creating a resource server, the other fields on the app creation page are not used.

        • “Redirects” is not used, but a value must be provided. You can use a URL associated with your service or a placeholder value like “https://localhost”.

        • “Scopes” are not relevant and make no difference, so this field should be left blank. The “Privacy Policy” and “Terms and Conditions” may be displayed to users making use of your action provider, but they are not required.

    • Make note of the “Client Id” in the expanded description of your app. This value will be used elsewhere in the creation of the service and is often referenced as client_id.

    • In the section “Client Secrets” click “Create a new secret”

      • Provide a name which is meaningful to you. It will not be displayed to other users.

      • Make note of the generated secret. Like the client_id this will be used later in development. Be sure not to lose it as it can only be displayed once. However, new client secrets can be created and old ones deleted at any time should the need for a replacement secret arise.

      • Set the client_id and client_secret on your command line to follow along with the rest of this guide.

        export CLIENT_ID=<client_id>
        export CLIENT_SECRET=<client_secret>
  2. Use the Globus Auth REST API to introspect your Action Provider Resource Server and create required Scopes.

    • Introspect the Globus Auth client to see the same settings you setup in the developer portal. Notice we exported the <client_id> and <client_secret> values generated during your registration on the Globus Developer Portal into environment variables.

    curl -s --user $CLIENT_ID:$CLIENT_SECRET https://auth.globus.org/v2/api/clients/$CLIENT_ID | jq
    • A successful return from this command is a JSON representation of the Globus Auth client similar to:

      {
      "client": {
          "scopes": [],
          "redirect_uris": [
          "https://localhost"
          ],
          "name": "My Action Provider",
          "links": {
          "privacy_policy": null,
          "terms_and_conditions": null
          },
          "grant_types": [
          "authorization_code",
          "client_credentials",
          "refresh_token",
          "urn:globus:auth:grant_type:dependent_token"
          ],
          "fqdns": [],
          "visibility": "private",
          "project": "a47b9014-9250-4e21-9de5-b4aac81d464b",
          "required_idp": null,
          "preselect_idp": null,
          "id": "8e98ba5a-21a9-4bef-ab6a-0fcdbed36405",
          "public_client": false,
          "parent_client": null
        }
      }
    • Of note is the field scopes. scopes are created to identify operations on the Action Provider. Typically, an Action Provide defines just one scope and it is provided to users in the Action Provider’s introspection (GET /) information in the field globus_auth_scope.

      • Creating a Scope:
        • Creation of a scope is required as the scope will be used in authenticating REST calls on the Action Provider.

        • Start by creating a “scope definition” JSON document in the following format replacing the name, description and optionally the scope_suffix

          {
              "scope": {
                  "name": "Action Provider Operations",
                  "description": "All Operations on My Action Provider",
                  "scope_suffix": "action_all",
                  "dependent_scopes": [{
                      "optional": false,
                      "requires_refresh_token": true,
                      "scope": "73320ffe-4cb4-4b25-a0a3-83d53d59ce4f"
                   }],
                  "advertised": true,
                  "allow_refresh_tokens": true
              }
          }
        • The name and description fields are purely informative and will be presented to other users who use the Globus Auth API to lookup the scope. The scope_suffix will be placed at the end of the generated “scope string” which is a URL identifier for the scope. It provides the context for the operations this scope covers among all operations your service provides. For Action Providers, we commonly use action_all to indicate all operations defined by the Action Provider API, but any string is acceptable.

        • dependent_scopes define scopes of other Globus Auth resource servers that your Action Provider will invoke to perform its work. For example, if your Action Provider uses Globus Transfer to first move some data to compute upon, the scope for the Globus Transfer service would be placed in the dependent_scopes list. In the most common case, as shown in the example, the scope for the Globus Groups API (with UUID 73320ffe-4cb4-4b25-a0a3-83d53d59ce4f) should be listed. This allows your Action Provider to determine what groups a user calling the provider belongs to and can therefore enforce policies, such as runnable_by or monitor_by based on group membership. If this scope is not listed as a dependent scope, the Action Provider Tools library will not be able to, and will therefore not attempt to, retrieve a user’s groups and so no policies based on Groups may be used. We encourage you to consult the Globus Auth Documentation for more information on creation and management of Scopes for more advanced scenarios such as other dependent Globus Auth based services such as Globus Transfer.

          Note: Scopes supplied in the dependent_scopes array must be identified by their UUID. The snippet below demonstrates how to lookup a scope’s UUID based on its uniquely idenfitfying FQDN

          # Target FQDN is https://auth.globus.org/scopes/actions.globus.org/transfer/transfer
          export SCOPE_STRING=https://auth.globus.org/scopes/actions.globus.org/transfer/transfer
          curl -s -u "$CLIENT_ID:$CLIENT_SECRET" "https://auth.globus.org/v2/api/scopes?scope_strings=$SCOPE_STRING" | jq ".scopes[0].id"
        • The advertised property indicates whether the scope will be visible to all users who do scope look ups on Globus Auth. You may select either true or false for this depending on your own policy. allow_refresh_tokens should generally be set to true, indicating that a client of the Action Provider who has authenticated the user via Globus Auth is allowed to refresh that authentication without further interactions from the user. Especially in the case where an Action may be long running and is monitored by an automated system like Globus Flows, it is important that token refresh is permitted.

        • With the scope creation JSON document complete, use the following REST interaction to create the scope in Globus Auth via the curl command

          curl -s --user "$CLIENT_ID:$CLIENT_SECRET" -H 'Content-Type: application/json' -XPOST https://auth.globus.org/v2/api/clients/$CLIENT_ID/scopes -d '<Insert Scope creation document from above>' | jq
        • This should return the definition of the new scope matching the values provided in your scope creation document. As an example:

          {
              "scopes": [
                  {
                      "dependent_scopes": [{
                          "optional": false,
                          "requires_refresh_token": true,
                          "scope": "73320ffe-4cb4-4b25-a0a3-83d53d59ce4f"
                       }],
                      "description": "<your description>",
                      "allows_refresh_token": true,
                      "client": "<client_id>",
                      "advertised": true,
                      "scope_string": "https://auth.globus.org/scopes/<client_id>/action_all",
                      "id": "<A UUID for this scope>",
                      "name": "<your scope name>"
                  }
              ]
          }
        • The returned scope_string, which always takes the form of a URL, will be the value exposed to users who wish to authenticate with Globus Auth to use your Action Provider. It will be part of the Action Provider description document, returned on the Action Provider Introspection operation (GET /) with the key globus_auth_scope.

        • Note that the returned value is an array of scopes. That is, more than one scope definition may be generated from the single scope creation request. This happens in the uncommon case where an FQDN has been registered for your client_id (refer to the Globus Auth Documentation for information on FQDN registration if you desire it, though it is not recommended). In this case, a similar scope definition will also be generated, but the scope_string will contain the FQDN value(s). The scope_string values may be used interchangeably both by users requesting authentication to the Action Provider and in the globus_auth_scope value of the Action Provider Description.

        • Check that the created scope(s) are correctly associated with the Action Provider:

          curl -s --user $CLIENT_ID:$CLIENT_SECRET https://auth.globus.org/v2/api/clients/$CLIENT_ID | jq
  3. Once your app and its scope(s) have been created and verified, remove your credentials from your command line environment. Be sure to take note of the client ID and its associated client secret for use other places in the toolkit.

    unset CLIENT_ID CLIENT_SECRET

Using the Toolkit

This toolkit provides the following components:

  1. Authentication helpers that make it easier to validate Globus Auth tokens and determine if a given request should be authorized

  2. An OpenAPI v3 specification and associated helpers that can be used to validate incoming requests and verify the responses your Action Provider generates. This document also defines the interface which must be supported by your REST API to have it function as an Action Provider.

  3. Simple bindings for the document types “Action Request” and “Action Status” to Python NamedTuple representations and helper functions for serializing and deserializing these structures to/from JSON.

  4. Helper methods for binding the REST API calls defined by the Action Interface to a Flask application. These helpers will perform the Authentication and Validation steps (as provided by components 1 and 2) and communicate with an Action Provider implementation using the structures defined in 3. For those users building an Action Provider using Flask, this provides a simplified method of getting the REST API implemented and removing common requirements so the focus can be on the logic of the Action provided.

Installation

Installation is via PyPi using, for example:

pip install globus-action-provider-tools

Authentication

The authentication helpers can be used in your action provider as follows:

from globus_action_provider_tools.authentication import TokenChecker
# You will need to register a client and scope(s) in Globus Auth
# Then initialize a TokenChecker instance for your provider:
checker = TokenChecker(
    client_id='YOUR_CLIENT_ID',
    client_secret='YOUR_CLIENT_SECRET',
    expected_scopes=['https://auth.globus.org/scopes/YOUR_SCOPES_HERE'],
)

When a request comes in, use your TokenChecker to validate the access token from the HTTP Authorization header.

access_token = request.headers['Authorization'].replace('Bearer ', '')
auth_state = checker.check_token(access_token)

The AuthState has several properties and methods that will make it easier for you to decide whether or not to allow a request to proceed:

# This user's Globus identities:
auth_state.identities
# frozenset({'urn:globus:auth:identity:9d437146-f150-42c2-be88-9d625d9e7cf9',
#           'urn:globus:auth:identity:c38f015b-8ad9-4004-9160-754b309b5b33',
#           'urn:globus:auth:identity:ffb5652b-d418-4849-9b57-556656706970'})
# Groups this user is a member of:
auth_state.groups
# frozenset({'urn:globus:groups:id:606dbaa9-3d57-44b8-a33e-422a9de0c712',
#           'urn:globus:groups:id:d2ff42bc-c708-460f-9e9b-b535c3776bdd'})

You’ll notice that both groups and identities are represented as strings that unambiguously signal what type of entity they represent. This makes it easy to merge the two sets without conflict, for situations where you’d like to work with a single set containing all authentications:

all_principals = auth_state.identities.union(auth_state.groups)

The AuthState object also offers a helper method, check_authorization() that is designed to help you test whether a request should be authorized:

resource_allows = ['urn:globus:auth:identity:c38f015b-8ad9-4004-9160-754b309b5b33']
auth_state.check_authorization(resource_allows)
# True

This method also accepts two special string values, 'public' and 'all_authenticated_users', together with keyword arguments that enable their use:

resource_allows = ['public']
auth_state.check_authorization(resource_allows, allow_public=True)
# True
resource_allows = ['all_authenticated_users']
auth_state.check_authorization(resource_allows, allow_all_authenticated_users=True)
# True

Caching

To avoid excessively taxing Globus Auth, the AuthState will, by default, cache identities and group memberships for 30 seconds.

The cache is initialized when you first instantiate your TokenChecker(). You should only need to create one TokenChecker instance for your application, and then you can re-use it to check each new token. In the event that you do need more than one TokenChecker, be aware that all TokenChecker instances in an app share the same underlying cache.

It is possible to customize a TokenChecker by supplying a custom configuration which gets passed on to the dogpile cache backend. Note that each new instance of a TokenChecker with a custom configuration will drop the cache and recreate it with the desired settings. Since all TokenCheckers share the same underlying cache, subsequent attempts to configure the cache will overwrite the previous cache’s settings and therefore only the last applied configuration will persist.

# Create TokenChecker with default settings
my_token_checker = TokenChecker(
    client_id=CLIENT_ID,
    client_secret=CLIENT_SECRET,
    expected_scopes=EXPECTED_SCOPES,
)

# Creating a TokenChecker with a custom config will drop the previous cache and
# create it with the new settings. Both TokenCheckers will use this new cache
new_token_checker = TokenChecker(
        client_id=config["client_id"],
        client_secret=config["client_secret"],
        expected_scopes=config["expected_scopes"],
        cache_config={
            "backend": "dogpile.cache.pylibmc",
            "timeout": "60",
            "url": ["127.0.0.1"],
        },
    )

Validation

There is an OpenAPI v3 specification for the Action Provider API available as described above. From this specification, we derive schemas that can be used to test incoming and outgoing messages. These schemas may be used to validate input documents and output documents within the service as follows.

from globus_action_provider_tools.validation import request_validator, response_validator, ValidationRequest
# Validating a request
request = ValidationRequest(provider_doc_type='ActionRequest', request_data=<input data>)
result = request_validator.validate(request)
# Or a response:
response = ValidationRequest(provider_doc_type='ActionStatus', request_data=<output data>)
result = response_validator.validate(response)
# get list of errors
errors = result.errors
# or get a single string summarizing all errors
err = result.error_msg

The request and response validation functions both take a ValidationRequest structure which has the name of the document type to be validated against and the data to be validated. At present, the document types supported are ActionRequest and ActionStatus documents as defined above.

Data Types

The toolkit provides some simple bindings for the document types defined by the Action Provider Interface to type-annotated Python3 NamedTuples. This can provide a convenient way to manipulate these document types within an Action Provider implementation. There is also a Python JSON Encoder provided which can be used with the built-in Python json package to properly encode these data types into JSON.

from globus_action_provider_tools.data_types import (
    ActionProviderDescription,
    ActionProviderJsonEncoder,
    ActionRequest,
    ActionStatus,
    ActionStatusValue,
)
status = ActionStatus(
    action_id=str(uuid.uuid4()),
    status=ActionStatusValue.SUCCEEDED,
    creator_id=caller_id,
    monitor_by=request.monitor_by,
    manage_by=request.manage_by,
    start_time=str(datetime.datetime.now().isoformat()),
    completion_time=str(datetime.datetime.now().isoformat()),
    release_after=60 * 60 * 24 * 30,  # 30-days in seconds
    display_status=ActionStatusValue.SUCCEEDED.name,
    details=result_details,
)
json_string = json.dumps(action_status, cls=ActionProviderJsonEncoder)

Flask Helper

As Action Providers are HTTP-servers, a common approach to building them is the use of the Flask framework. To aid in developing Flask-based Action Providers, helper methods are provided which encapsulate much of the other functionality in the framework: authentication, validation and serialization for easy use in a Flask-based application. Rather than defining each of the Action Provider Interface routes in the Flask application, helpers are provided which declare the necessary routes to Flask, perform the serialization, validation and authentication on the request, and pass only those requests which have satisfied these conditions on to a user-defined implementation of the routes.

To use the helpers, you must define functions corresponding to the various methods of the Action Provider interface (run, status, release, cancel), and must provide the Action Provider introspection information in an instance of the ActionProviderDescription NamedTuple class defined in the tookit’s data_types package. The application must also provide a Flask blueprint object to which the toolkit can attach the new routes. It is recommended that the blueprint be created with a url_prefix so that the Action Provider Interface routes are rooted at a distinct root path in the application’s URL namespace.

A brief example of setting up the flask helper is provided immediately below. A more complete example showing implementation of all the required functions is provided in the Appendix. It is appropriate to use the skeleton in the Appendix as a starting point for any new Action Providers which are developed.

from globus_action_provider_tools.data_types import (
    ActionProviderDescription,
    ActionRequest,
    ActionStatus,
    ActionStatusValue,
)
from globus_action_provider_tools.flask import (
    ActionStatusReturn,
    add_action_routes_to_blueprint,
)
action_blueprint = Blueprint("action", __name__, url_prefix="/action")
provider_description = ActionProviderDescription(
    globus_auth_scope="<scope created in Globus Auth>",
    title="My Action Provider",
    admin_contact="support@example.com",
    synchronous=True,
    input_schema={}, # JSON Schema representation of the input on the request
    log_supported=False
)
add_action_routes_to_blueprint(
    action_blueprint,
    CLIENT_ID,
    CLIENT_SECRET,
    CLIENT_NAME,
    provider_description,
    action_run,
    action_status,
    action_cancel,
    action_release,
)

In this example, the values CLIENT_ID, CLIENT_SECRET and CLIENT_NAME are as defined in Globus Auth as described above (where CLIENT_NAME is almost always passed as None except in the uncommon, legacy case where a particular name has been associated with a Globus Auth client). The values action_run, action_status, action_cancel and action_release are all functions which will be called by the framework when the corresponding HTTP requests are called. Where appropriate, these functions are implemented in terms of the toolkit’s data types so the need for JSON serialization and deserialization is greatly reduced from the application code. The framework will also provide validation of input ActionRequest data to the /run method prior to invoking the action_run function. As long as the return value from the various functions is of type ActionStatus, the framework will also insure that the returned JSON data conforms to the Action Provider Interface. The example in the Appendix demonstrates how these functions can be implemented.

Appendix: Example Action Provider

import uuid
import datetime
from typing import Dict, Optional, Tuple

from flask import Flask, Blueprint
from werkzeug.exceptions import Conflict, NotFound

from globus_action_provider_tools.data_types import (
    ActionProviderDescription,
    ActionRequest,
    AuthState,
    ActionStatus,
    ActionStatusValue,
)
from globus_action_provider_tools.flask import (
    ActionStatusReturn,
    add_action_routes_to_blueprint,
)

skeleton_blueprint = Blueprint("skeleton", __name__, url_prefix="/skeleton")

input_schema = {
    "$id": "https://automate.globus.org/skeleton_action_provider.input.schema.json",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Skeleton Action Provider Input Schema",
    "type": "object",
    "properties": {"input_string": {"type": "string"}},
    "additionalProperties": False,
    "required": ["input_string"],
}

provider_description = ActionProviderDescription(
    globus_auth_scope="https://auth.globus.org/scopes/16e16447-209a-4825-ae19-25e279d91642/action_all_with_groups",
    title="skeleton_action_provider",
    admin_contact="support@globus.org",
    synchronous=True,
    input_schema=input_schema,
    log_supported=False,  # This provider doesn't implement the log callback
)

# A simulated database mapping input user action requests identifiers to a previously
# seen request id and the corresponding action id
_fake_request_db: Dict[str, Tuple[ActionRequest, str]] = {}

_fake_action_db: Dict[str, ActionStatus] = {}

def _authorize_action_access(
    action_id: str, auth_state: AuthState, policy_type: str = "manage_by") -> ActionStatus:
    status = _fake_action_db.get(action_id)
    if status is None:
        raise NotFound(f"No Action with id {action_id}")
    if policy_type == "manage_by":
        policy_group = status.manage_by
    elif policy_type == "monitor_by":
        policy_group = status.monitor_by
    if policy_group is None:
        policy_group = []
    allowed_set = set([status.creator_id] + [status.monitor_by])
    if auth_state.check_authorization(allowed_set, allow_all_authenticated_users=True):
        return status
    else:
        raise NotFound(f"No Action with id {action_id}")

def action_run(request: ActionRequest, auth: AuthState) -> ActionStatusReturn:
    caller_id = auth.effective_identity
    request_id = request.request_id
    full_request_id = f"{caller_id}:{request_id}"
    prev_request = _fake_request_db.get(full_request_id)
    if prev_request is not None:
        if prev_request[0] == request:
            return action_status(prev_request[1], auth)
        else:
            raise Conflict(
                f"Request with id {request_id} already present with different parameters "
            )
    # Local processing would happen here
    result_details = {
        # This is safe because the input has been validated against the input schema
        "you_said": request.body["input_string"]
    }
    status = ActionStatus(
        action_id=str(uuid.uuid4()),
        status=ActionStatusValue.SUCCEEDED,
        creator_id=caller_id,
        monitor_by=request.monitor_by,
        manage_by=request.manage_by,
        start_time=str(datetime.datetime.now().isoformat()),
        completion_time=str(datetime.datetime.now().isoformat()),
        release_after=request.release_after or "P30D",
        display_status=ActionStatusValue.SUCCEEDED.name,
        details=result_details,
    )
    _fake_request_db[full_request_id] = (request, status.action_id)
    _fake_action_db[status.action_id] = status
    return status

def action_status(action_id: str, auth: AuthState) -> ActionStatusReturn:
    return _authorize_action_access(action_id, auth, "monitor_by"), 200

def action_cancel(action_id: str, auth: AuthState) -> ActionStatusReturn:
    status = _authorize_action_access(action_id, auth, "manage_by")
    if status.status in (ActionStatusValue.SUCCEEDED, ActionStatusValue.FAILED):
        return status
    # Process Action cancellation
    status.status = ActionStatusValue.FAILED
    status.display_status = "Canceled by user request"
    return status

def action_release(action_id: str, auth: AuthState) -> ActionStatusReturn:
    status = _authorize_action_access(action_id, auth, "manage_by")
    if status.status not in (ActionStatusValue.SUCCEEDED, ActionStatusValue.FAILED):
        raise Conflict("Action is not complete")
    _fake_action_db.pop(action_id)
    # Both fake and badly inefficient
    remove_req_id: Optional[str] = None
    for req_id, req_and_action_id in _fake_request_db.items():
        if req_and_action_id[1] == action_id:
            remove_req_id = req_id
            break
    if remove_req_id is not None:
        _fake_request_db.pop(remove_req_id)
    return status, 200

def main():
    app = Flask(__name__)
    app.url_map.strict_slashes = False
    add_action_routes_to_blueprint(
        skeleton_blueprint,
        CLIENT_ID,
        CLIENT_SECRET,
        None,
        provider_description,
        action_run,
        action_status,
        action_cancel,
        action_release,
    )
    app.register_blueprint(skeleton_blueprint)
    app.run(debug=True)

if __name__ == "__main__":
    main()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

globus-action-provider-tools-0.9.1.tar.gz (57.0 kB view hashes)

Uploaded Source

Built Distribution

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page