WarpZone SDK
This package contains tools used in the WarpZone project. These tool include:
Client for Storage
Blob storage
WarpzoneBlobClient client is used for uploading to and downloading from Azure Storage Blob Service.
Client for Servicebus
Due to limitations on message sizes, we use different methods for sending events and data using Azure Service Bus.
Events
We use the Service Bus for transmitting event messages. By an event, we mean a JSON formatted message, containing information about an event occuring in one part of the system, which needs to trigger another part of the system (such as an Azure Function trigger).
WarpzoneEventClient client is used for sending and receiving events.
--
Data
We do not use the Service Bus for transmitting data directly. Instead, we use a claim-check pattern, were we store the data using Storage Blob, and transmit an event about the details of this stored data.
WarpzoneDataClient client is used for sending and receiving data in this way. The following diagram shows how the process works:
- Data is uploaded
- Event containing the blob location is send
- Event is received
- Data is downloaded using the blob location contained in the event
The transmitted event has the following format:
{
"container_name": "<container-name>",
"blob_name": "<blob-name>",
"timestamp": "<%Y-%m-%dT%H:%M:%S%z>"
}
The data will be stored with
<container-name>=<topic-name><blob-name>=<subject>/year=<%Y>/month=<%m>/day=<%d>/hour=<%H>/<message-id>.<extension>
Function Wrapper
For executing logic, we use a framework built on top of Azure Functions. The following diagram shows how the framework works:
- The function is triggered by a trigger object (e.g. a timer or a message being received)
- Possible dependency objects are initialized (potentially using information from the trigger). These are used to integrate with external systems (e.g. a database client).
- Using the trigger and dependencies as inputs, the function outputs and an output object (e.g. a message being sent).
The reason we have used our own framework instead of Azure Functions directly, is that we want to use our own objects as triggers, dependencies and outputs, instead of the built-in bindings. For example, as explained above, we have created our own abstraction of a message for transmitting data (warpzone.DataMessage); so we would like to use this, instead of the built-in binding azure.function.ServiceBusMessage.
Since it is not yet possible to define custom bindings in Python, we have defined our own wrapping logic, to handle the conversion between our own objects and the built-in bindings. The following diagram shows how the wrapping logic works:
- Azure trigger binding is converted to trigger object
- Either
- (a) Output object is converted to Azure output binding.
- (b) Use custom output logic, when no suitable output binding exists (e.g. we use the Azure Service Bus SDK instead of the Service Bus output binding, since this is recommended)
- All logs and traces are sent to App Insights automatically.
--
Examples
Azure Function with data messages as trigger and output:
# function.json
{
"scriptFile": "__init__.py",
"entryPoint": "main",
"bindings": [
{
"name": "msg",
"type": "serviceBusTrigger",
"direction": "in",
"connection": "...",
"topicName": "...",
"subscriptionName": "..."
}
]
}
import warpzone as wz
def do_nothing(data_msg: wz.DataMessage) -> wz.DataMessage:
return data_msg
main = wz.functionize(
f=do_nothing,
trigger=wz.triggers.DataMessageTrigger(binding_name="msg"),
output=wz.outputs.DataMessageOutput(wz.Topic.UNIFORM)
)
Azure Function with HTTP messages as trigger and output:
# function.json
{
"scriptFile": "__init__.py",
"entryPoint": "main",
"bindings": [
{
"authLevel": "anonymous",
"name": "req",
"type": "httpTrigger",
"direction": "in"
},
{
"type": "http",
"direction": "out",
"name": "$return"
}
]
}
# __init__.py
import warpzone as wz
import azure.functions as func
def return_ok(req: func.HttpRequest) -> func.HttpResponse:
return func.HttpResponse("OK")
main = wz.functionize(
f=return_ok,
trigger=wz.triggers.HttpTrigger(binding_name="req"),
output=wz.outputs.HttpOutput()
)
Azure Function using dependencies:
import warpzone as wz
def do_nothing(
data_msg: wz.DataMessage,
db: wz.WarpzoneDatabaseClient,
) -> wz.DataMessage:
return data_msg
main = wz.functionize(
f=do_nothing,
trigger=wz.triggers.DataMessageTrigger(binding_name="msg"),
output=wz.outputs.DataMessageOutput(wz.Topic.UNIFORM),
dependencies=[wz.dependencies.DeltaDatabaseDependency()],
)
Release files for warpzone-sdk 19.0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| warpzone_sdk-19.0.2.tar.gz | 21.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| warpzone_sdk-19.0.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 51.2 kB
Release files / warpzone_sdk-19.0.2.tar.gz
| Download URL | warpzone_sdk-19.0.2.tar.gz |
|---|---|
| Size | 21.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
38699d44adcb53c1632f2c86d22e5b09406fab7c0b36b563b84a034477b2baa3
|
|
BLAKE2b-256 checksum How to use checksums |
59804cfdaa007d88d8eb03e5968ee40c21af053300bbe1a11059b44bfbecfd47
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.3.4 CPython/3.10.20 Linux/6.17.0-1010-azure
|
Release files / warpzone_sdk-19.0.2-py3-none-any.whl
| Download URL | warpzone_sdk-19.0.2-py3-none-any.whl |
|---|---|
| Size | 30.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7a93ec7c36367e75ede411ab6ad065f6b1bec245a690f5dfac9f0617530ae8d9
|
|
BLAKE2b-256 checksum How to use checksums |
836999bd6d33b2fd44c6d06f3f295829937b70d8404af5840e8dd68b1cd3d9c6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.3.4 CPython/3.10.20 Linux/6.17.0-1010-azure
|
