Skip to main content

The LSST SQuaRE framework for Roundtable bots.

Project description

Safir is the SQuaRE Framework for Roundtable

Roundtable is the SQuaRE application and bot deployment platform, hosted on Kubernetes with Argo CD. Safir is a Python package that lets you develop Roundtable bots, based on the FastAPI asyncio web framework.

Install from PyPI:

pip install safir

The best way to create a new Safir-based Roundtable bot is with the fastapi_safir_app template.

Read more about Safir at https://safir.lsst.io

Features

  • Set up an httpx.AsyncClient as part of the application start-up and shutdown lifecycle.
  • Set up structlog-based logging.
  • Middleware for attaching request context to the logger to include a request UUID, method, and route in all log messages.
  • Process X-Forwarded-* headers to determine the source IP and related information of the request.
  • Gather and structure standard metadata about your application.
  • Operate a distributed Redis job queue with arq using convenient clients, testing mocks, and a FastAPI dependency.

Developing Safir

The best way to start contributing to Safir is by cloning this repository creating a virtual environment, and running the init command:

git clone https://github.com/lsst-sqre/safir.git
cd safir
make init

For details, see https://safir.lsst.io/dev/development.html.

Change log

3.2.0 (2022-05-13)

  • New support for arq, the Redis-based asyncio distributed queue package. The safir.arq module provides an arq client and metadata/result data classes with a mock implementation for testing. The FastAPI dependency, safir.dependencies.arq.arq_dependency, provides a convenient way to use the arq client from HTTP handlers.

3.1.0 (2022-06-01)

  • Add new FastAPI middleware CaseInsensitiveQueryMiddleware to aid in implementing the IVOA protocol requirement that the keys of query parameters be case-isensitive.

3.0.3 (2022-05-16)

  • Correctly handle the possibility that request.client is None.

3.0.2 (2022-03-25)

  • Fix handling of passwords containing special characters such as @ and / in safir.database.

3.0.1 (2022-02-24)

  • safir.database retains the port in the database URL, if provided.

3.0.0 (2022-02-23)

  • XForwardedMiddleware no longer sets forwaded_proto in the request state and instead directly updates the request scope so that subsequent handlers and middleware believe the request scheme is that given by an X-Forwarded-Proto header. This fixes the scheme returned by other request methods and attributes such as url_for in cases where the service is behind an ingress that terminates TLS.
  • Add new FastAPI dependencies auth_dependency and auth_logger_dependency from the safir.dependencies.gafaelfawr module. auth_dependency returns the username of the user authenticated via Gafaelfawr (pulled from the X-Auth-Request-User header. auth_logger_dependency returns the same logger as logger_dependency but with the user parameter bound to the username from auth_dependency.
  • Add utility functions to initialize a database and create a sync or async session. The session creation functions optionally support a health check to ensure the database schema has been initialized.
  • Add new FastAPI dependency db_session_dependency that creates a task-local async SQLAlchemy session.
  • Add utility functions datetime_from_db and datetime_to_db to convert between timezone-naive UTC datetimes stored in a database and timezone-aware UTC datetimes used elsewhere in a program.
  • Add a run_with_async decorator that runs the decorated async function synchronously. This is primarily useful for decorating Click command functions (for a command-line interface) that need to make async calls.

2.4.2 (2022-01-24)

  • Add a very basic and limited implementation of list_namespaced_pod to safir.testing.kubernetes.

2.4.1 (2022-01-14)

  • In the Kubernetes mock API, change the expected body for patch_namespaced_custom_object_status to reflect the need to send a JSON patch for the entirety of the /status path in order to work around a kubernetes_asyncio bug.

2.4.0 (2022-01-13)

  • Add an initialize_kubernetes helper function to load Kubernetes configuration. Add the safir.testing.kubernetes module, which can be used to mock the Kubernetes API for testing. To use the new Kubernetes support, depend on safir[kubernetes].

2.3.0 (2021-12-13)

  • When logging in JSON format, use severity instead of level for the log level for compatibility with Google Log Explorer.
  • In the FastAPI logger_dependency, add log information about the incoming requst in a format compatible with Google Log Explorer.

2.2.0 (2021-11-09)

  • Restore previous http_client_dependency behavior by enabling following redirects by default. This adjusts for the change of defaults in httpx 0.20.0.

2.1.1 (2021-10-29)

  • Require structlog 21.2.0 and adjust logger configuration of exception handling for the expectations of that version.

2.1.0 (2021-08-09)

  • Add safir.models.ErrorModel, which is a model of the error message format preferred by FastAPI. Using the model is not necessary but it’s helpful to reference it in API documentation to generate more complete information about the error messages.
  • Mark all FastAPI dependencies as async so that FastAPI doesn’t run them in an external thread pool.

2.0.1 (2021-06-24)

  • Defer creation of httpx.AsyncClient until the first time it is requested. Allow re-creation after aclose(), which makes it easier to write test suites.

2.0.0 (2021-06-21)

As of this release, Safir is a helper library for FastAPI applications instead of aiohttp applications. Much of the library has changed. Authors of software that uses Safir should read the documentation again as part of the upgrade to FastAPI.

Included in this release is:

  • A FastAPI dependency to provide a structlog logger as configured by the safir.logging package, replacing the aiohttp middleware.
  • A FastAPI dependency to provide a global httpx.AsyncClient, replacing the middleware that provided an aiohttp client.
  • Starlette (FastAPI) middleware to parse X-Forwarded-* headers and update request information accordingly.
  • safir.metadata.get_metadata now returns a Pydantic model.

As of this release, Safir only supports Python 3.8 or later.

1.0.0 (2021-06-18)

Safir v1 will be the major release supporting aiohttp. Starting with Safir v2, Safir will become a support library for FastAPI applications.

This release has no significant difference from 0.2.0. The version bump is to indicate that the aiohttp support is stable.

0.2.0 (2021-06-10)

  • configure_logging now supports an optional add_timestamp parameter (false by default) that adds a timestamp to each log message.
  • Update dependencies.

0.1.1 (2021-01-14)

  • Fix duplicated log output when logging is configured multiple times.
  • Update dependencies.

0.1.0 (2020-02-26)

  • The first release of Safir featuring:
    • safir.http for adding an aiohttp.ClientSession to your application.
    • safir.logging for configuring structlog loggers.
    • safir.metadata helps your gather and structure metadata about your application for publishing on metadata endpoints.
    • safir.middleware includes a logging middleware that adds a logger with bound context about the request to your Request object.
    • Documentation about these features and a tutorial for starting a new application with the roundtable_aiohttp_bot template.

MIT License

Copyright (c) 2020 Association of Universities for Research in Astronomy, Inc. (AURA)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

safir-3.2.0.tar.gz (66.1 kB view hashes)

Uploaded source

Built Distribution

safir-3.2.0-py3-none-any.whl (31.0 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page