Skip to main content

Hedwig Python Library

Project description

https://github.com/cloudchacho/hedwig-python/workflows/Pytest/badge.svg https://codecov.io/gh/cloudchacho/hedwig-python/branch/main/graph/badge.svg?token=81LqiGysSn https://img.shields.io/pypi/v/authedwig.svg?style=flat-square https://img.shields.io/pypi/pyversions/authedwig.svg?style=flat-square https://img.shields.io/pypi/implementation/authedwig.svg?style=flat-square https://img.shields.io/badge/code%20style-black-000000.svg

Hedwig is a inter-service communication bus that works on AWS SQS/SNS, while keeping things pretty simple and straight forward. It uses json schema draft v4 for schema validation so all incoming and outgoing messages are validated against pre-defined schema.

Hedwig allows separation of concerns between consumers and publishers so your services are loosely coupled, and the contract is enforced by the schema validation. Hedwig may also be used to build asynchronous APIs.

For intra-service messaging, see Taskhawk.

Only Python 3.6+ is supported currently.

You can find the latest, most up to date, documentation at Read the Docs.

Quick Start

First, install the library:

$ pip install authedwig[aws,jsonschema]

Next, set up a few configuration settings:

Common required settings:

HEDWIG_QUEUE = "DEV-MYAPP"

HEDWIG_CALLBACKS = {
    ("email.send", "1.*"): "send_email",
}

HEDWIG_MESSAGE_ROUTING = {
    ("email.send", "1.*"): "send-email-v1",
}

HEDWIG_JSONSCHEMA_FILE = "schema.json"

When using AWS, additional required settings are:

AWS_ACCESS_KEY = <YOUR AWS KEY>
AWS_ACCOUNT_ID = <YOUR AWS ACCOUNT ID>
AWS_REGION = <YOUR AWS REGION>
AWS_SECRET_KEY = <YOUR AWS SECRET KEY>

HEDWIG_CONSUMER_BACKEND = 'hedwig.backends.aws.AWSSQSConsumerBackend'
HEDWIG_PUBLISHER_BACKEND = 'hedwig.backends.aws.AWSSNSPublisherBackend'

In case of GCP, additional required settings are:

HEDWIG_CONSUMER_BACKEND = 'hedwig.backends.gcp.GooglePubSubConsumerBackend'
HEDWIG_PUBLISHER_BACKEND = 'hedwig.backends.gcp.GooglePubSubPublisherBackend'

HEDWIG_SUBSCRIPTIONS = ["dev-user-created-v1"]

If running outside Google Cloud (e.g. locally), set GOOGLE_APPLICATION_CREDENTIALS.

Within Google Cloud, these credentials and permissions are managed by Google using IAM.

If the Pub/Sub resources lie in a different project, set GOOGLE_CLOUD_PROJECT to the project id.

For Django projects, simple use Django settings to configure Hedwig. For Flask projects, use Flask config. For other frameworks, you can either declare an environment variable called SETTINGS_MODULE that points to a module where settings may be found, or manually configure using hedwig.conf.settings.configure_with_object.

Create a JSON-schema and save as schema.json:

{
    "id": "https://github.com/cloudchacho/hedwig-python/schema#",
    "$schema": "http://json-schema.org/draft-04/schema",
    "schemas": {
        "email.send": {
            "1.*": {
                "description": "Request to send email",
                "type": "object",
                "required": [
                    "to",
                    "subject"
                ],
                "properties": {
                    "to": {
                        "type": "string",
                        "pattern": "^\\S+@\\S+$"
                    },
                    "subject": {
                        "type": "string",
                        "minLength": 2
                    }
                }
            }
        }
    }
}

Then, simply define your topic handler:

def send_email(message: hedwig.Message = None) -> None:
    # send email

And finally, send a message:

message = hedwig.Message.new(
    "email.send",
    StrictVersion('1.0'),
    {
        'to': 'example@email.com',
        'subject': 'Hello!',
    },
)
message.publish()

Development

Getting Started

Assuming that you have Python, pyenv and pyenv-virtualenv, and protoc installed, set up your environment and install the required dependencies like this instead of the pip install authedwig defined above:

$ git clone https://github.com/cloudchacho/hedwig.git /usr/local/lib/protobuf/include/hedwig
...
$ git clone https://github.com/cloudchacho/hedwig-python.git
$ cd hedwig-python
$ pyenv virtualenv 3.6.5 hedwig-3.6
...
$ pyenv activate hedwig-3.6
$ pip install -r requirements/dev-3.6.txt

Re-compile protobuf

On making any change to test protobufs or container protobuf, the file would need to be re-compiled:

$ make proto_compile

Running Tests

You can run tests in using make test. By default, it will run all of the unit and functional tests, but you can also specify your own py.test options.

$ py.test
$ py.test tests/test_consumer.py

Generating Documentation

Sphinx is used for documentation. You can generate HTML locally with the following:

$ pip install -e .[dev]
$ make docs

Getting Help

We use GitHub issues for tracking bugs and feature requests.

  • If it turns out that you may have found a bug, please open an issue

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

authedwig-9.3.1.tar.gz (63.3 kB view details)

Uploaded Source

Built Distribution

authedwig-9.3.1-py3-none-any.whl (48.3 kB view details)

Uploaded Python 3

File details

Details for the file authedwig-9.3.1.tar.gz.

File metadata

  • Download URL: authedwig-9.3.1.tar.gz
  • Upload date:
  • Size: 63.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for authedwig-9.3.1.tar.gz
Algorithm Hash digest
SHA256 08ab51e1282197243fd34c4f8a4de57cb51db6f1ee2dd0849a98506d66076d83
MD5 a1145ebcce67ca565f562698b596d4f5
BLAKE2b-256 38de441b08a914751a4a0eda9745de1c0fb3f7289654077109754c350ea0e791

See more details on using hashes here.

File details

Details for the file authedwig-9.3.1-py3-none-any.whl.

File metadata

  • Download URL: authedwig-9.3.1-py3-none-any.whl
  • Upload date:
  • Size: 48.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for authedwig-9.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 c891c249cdeabc2efd3bdd36233804c0f58c2e9457264410f308f91d999376be
MD5 f01fbdcbe4a34b7eb8f3dbd10065650a
BLAKE2b-256 8542bc162ee42fc9b41c0b367d484249114f5914f0b4c62356fe6f16285d2c0e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page