Libraries for developing the arivo openmodule
Project description
OpenModule V2
Some additional documentation:
Changes
Breaking changes are annotated here.
To quickly check if your service is susceptible to a known issue have a look here.
Coding Standard
For ARIVO developers we have defined a simple coding standard here
Features
The openmodule package provides a lot of features:
Settings
The openmodule package uses a global lazy configuration openmodule.config.settings
. This setting includes some
standard parameters defined in openmodule.config.GlobalSettings
and parameters from a customizable module. To specify
the module you can call settings.configure(module)
or you can set the environment variable SETTINGS_MODULE
. Per
default settings looks for the config
module (it also looks for the tests/config
module first for test cases)
Models
Inherit from OpenModuleModel
or in case of ZMQ messages from ZMQMessage
. Models use
pydantic (docs), check openmodule.models.* for some examples (e.g.
PresenceBaseMessage for alias)
Core
The base of the new openmodule, every package should have exactly one. The core handles various things:
- sentry
- logging
- dsvgo
- messaging
- health
- alerting
- database
core = init_openmodule(config, **kwargs)
shutdown_openmodule()
Messaging
Receiving messages
The core handles message distribution with a dispatcher. You only need to register your callback.
- register_schema: Automatically create a schema for your message handler and its models -> Beware that you need to document your handler method
core.messages.register_handler(b"topic", MessageClass, callback, filter={type = "demo"}, register_schema = True)
It may also be used together with an event listener to provide further functionality
event_listener = EventListener(log=logger)
core.messages.register_handler(b"topic", MessageClass, event_listener, filter={type = "demo"})
...
event_listener.append(some_function)
Sending messages
It is even easier to send messages
message = ZMQMessage(name=core.config.NAME, type="demo")
core.publish(message, b"topic")
Health
Due to the new convention, the health message should only represent if the service is still alive. This is done automatically by the core. If you need to specify some meta data or errors you can pass your handler to the core or set it later
def healthy() -> HealthResult:
if error:
return health_error("we have an error", meta=dict(error="error"))
return health_ok(meta=dict(this="is_easy"))
core = init_openmodule(config, health_handler=healthy)
# or
core.health.health_hanlder = healthy
Alerting
The new core also includes an alert handler.
core.alerts.send(...)
alert_id = core.alerts.get_or_add_alert_id(...)
core.alerts.send_with_alert_id(alert_id, ...)
Database
The openmodule package now also feature a simple database which can be also specified during the template creation. If you missed it there, just copy the directory src/database from the template. For more infos see here
RPCs
A new RPC server/client was implemented. It works like before and also includes better filtering:
- if a channel is provided for a filter, only rpcs of that channel will be subject to that filter
- if a type is provided for a filter, only rpcs of that type will be subject to that filter
- register_schema: Automatically create a schema for your rpc and its models -> Beware that you need to document your handler method
def handler(request: AccessRequest):
"""
awesome description
"""
rpc = RPCServer(config=core.config, context=core.context)
rpc_server.add_filter(self._backend_filter, "backend", "auth")
rpc_server.register_handler("backend", "auth", request_class=AccessRequest,
response_class=AccessResponse, handler=handler, register_schema=True)
rpc.run()
Utils
Api (DEPRECATED)
We implemented a very basic Api class you can use for http request and that handles errors and authentication. Either inherit it or create a class.
api = Api(**kwargs)
try:
res = api.post("some_url", payload=stuff)
except ApiException as e:
if e.retry: # <- makes sense to try again - timeouts or server not available ...
...
Backend
There is also a basic implementation of a backend that provides registration and message passing.
class MyBackend(Backend):
def check_access(self, request: AccessRequest) -> List[Access]:
...
def check_in(self, message: CountMessage):
...
def check_out(self, message: CountMessage):
...
Charset
Useful functions for character manipulation
Connection Status
Helper class that checks the connection status of ogclient to our server:
connection_status = ConnectionStatusListener(core.messages)
connection_status.on_connect.append(some_function)
connection_status.on_disconnect.append(some_function)
Matching
Useful functions for license plate matching
Presence
Helper class for listening to presence messages.
presence_listener = PresenceListener(core.messages)
presence_listener.on_enter.append(some_function)
Package Reader
Helper class to read settings (env, yml) of specified services. The services are only considered valid and parsed if a revision file for the service exists. Functions:
- installed_services: Returns a list of installed services, based on prefix
- load_setting: Loads the setting of a single service
- load_with_service_prefix: Loads the setting of all services starting with the prefix (no or empty prefix means no filter)
- load_with_hardware_type_prefix: Load the setting of all services with the given hardware type (i.e.
cam-ip
) - load_with_parent_type_prefix: Load the setting of all services with the given parent type (i.e.
cam-ip
)
Bridged Slave Detection
Some services behave differently if they are started on a bridged slave device. The function is_bridged_slave()
is a helper function to detect this. To use this function on the device, you need to mount the dist folder (/opt/openmodule/dist2).
The function returns:
- None: No dist directory, no bridge service installed, multiple bridges installed
- True: Bridge service is installed and
MASTER
is set there - False: Bridge service is installed and
MASTER
is not set there
You can also set the env variable BRIDGED_SLAVE
to True/False directly, so you do not need to care about the service settings.
This is useful for testing (@override_settings(BRIDGED_SLAVE=True)) or running local (set env var BRIDGED_SLAVE)
Anonymization
The openmodule framework uses rpc requests and messages to trigger the anonymization of data.
- Message: You can send a AnonymizeMessage (topic:
privacy
). The message includes a session_id and vehicle_ids to delete - RPC Request: You can send an AnonymizeRequest with channel=
privacy
, type=anonymize
to the DSGVO container. This session only includes session_ids. The DSGVO container will then match vehicle_ids to the session_ids and redistribute the request with the prior mentioned message.
A container with sensible data then needs to implement the message listener for the privacy messages:
Example:
core.messages.register("privacy", AnonymizeMessage, anonymize_data)
def anonymize_data(message: AnonymizeMessage):
for vid in message.vehicle_ids:
delete_vehicle_image_by_vehicle_id(vid)
IMPORTANT You still have to take care of data retention in each service separately, meaning you have to delete data independently of these anonymization messages. i.e. the DSGVO service deletes data if we need disk space or the eventlog deletes events after 30 days by default
Documentation
Openmodule >= 3.0.5 features automatic generation of Rpc and Message Schemas including their models. The generation uses data that is generated during the test runs to create an OpenApi Schema. Your RPCs and Message handlers are automatically documented if:
- You use the message dispatcher of the core (OpenModuleCoreTestMixin)
- You use the RPCServer of Openmodule
You can also register models yourself if you want them documented, but you may need to save the Schema in this case:
from openmodule.utils.schema import Schema
Schema.save_model(Model)
Schema.save_rpc(channel, type, request, reqponse, handler)
Schema.save_message(topic, message_class, handler, filter)
Schema.to_file()
With default parameters, you need to document your handler functions with a doc string, that is then included as a description.
Testing
A separate package for testing openmodule packages exists within openmodule - openmodule-test. For more infos see here
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openmodule-9.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c68ecb32df4f59278b6417e552c7aec78bd3b93002b6a85f46faddf3909066fc |
|
MD5 | 8d5fa0aa96fdd616df2ab81ae91c8c35 |
|
BLAKE2b-256 | 767f0f6bb00314c73a364e978849f7a1101806fcfb11206b5d58380fee8068be |