euclid-pdk

Python client library for the euclid server.

It covers the three things everything else needs - the connection, request signing, and EAM, euclid's access management module - and the eleven modules an application spends its time in: ESM (storage), EQS (queues), ENS (topics), EES (events), EKM (keys), ESS (secrets), EKV (key-value tables), EAG (the API gateway), EAP (the applications behind it), ETS (FTP and SFTP onto a bucket) and EMO (monitoring). The remaining modules (EMM, EMD and the rest) speak the same protocol over the same client and will follow.

Requires Python 3.10 or newer, and has no dependencies. Installing this SDK does not bring a TLS stack, an HTTP client and a JSON parser along with it: the wire protocol is JSON over HTTP and the signatures are HMAC-SHA256, all of which the standard library already covers.

Installation

pip install euclid-pdk

From a checkout:

pip install -e ".[dev]"

Usage

Log in once and reuse the session:

from euclid import Euclid

session = Euclid.for_server("https://euclid.example.com").login("jens", "secret")

for user in session.list_users(prefix="j", page_size=25).users:
    print(user.user_id, user.email)

Every other module hangs off that session, one client each and one connection each:

esm, eqs, ess = session.esm(), session.eqs(), session.ess()

bucket = esm.create_bucket("reports")
esm.upload_file(bucket.ern, "2026/q3.pdf", "q3.pdf")

queue = eqs.create_queue("orders")
eqs.send_message(queue.ern, '{"order": 17}')

for message in eqs.receive_messages(queue.ern, wait_time=20).messages:
    handle(message.body)
    eqs.delete_message(message.receipt_handle)

connect(password=ess.get_secret("db-password").value)

The builder form takes the same options one at a time, which reads better when there are several:

from euclid import Euclid, SigningScheme

session = (Euclid.for_server("https://euclid.example.com")
           .access()
           .credentials("jens", "secret")
           .namespace("development")
           .signing_scheme(SigningScheme.RFC9421)
           .ca_cert_path("/etc/euclid/euclid_cert.crt")
           .login())

A session holds a connection, so close it when you are done - or use it as a context manager:

with Euclid.for_server(url).login("jens", "secret") as session:
    session.create_account("111", "acme", "an account")

Accounts and namespaces

Every named thing in euclid lives in an account and a namespace, and is unique only within that pair: two namespaces may each have an orders queue, a suppliers table, a billing application or an orders route, and they have nothing to do with each other. A bare name always means yours — get_queue_ern("orders"), a bucket named in a deployment, a queue named in a grant — and the server resolves it in the account the session logged into and the namespace it is scoped to. It cannot reach into another namespace's resource of the same name.

That scope is the session's, and change_namespace moves it:

session.change_namespace("development")
orders = session.eqs().get_queue_ern("orders")   # development's orders queue

Two consequences worth knowing. Resources that run as processes — EAP applications and ETS transfer servers — also carry a runtime_name, because a process, its socket and its log channel have no namespace to live in and so cannot be keyed by an ID two namespaces may share; it is issued once and never changes, so moving an application does not orphan what is already running. And for EAP, changing an application's namespace is a move rather than a field change: the buckets and queues it may reach are re-resolved in the namespace it moves to, and the move is refused if an application of that ID already lives there.

The two calls that empty a whole namespace — eqs.purge_all_queues() and ens.purge_all_topics() — default to the session's own, which is the scope the caller can see. Emptying the field asks for every namespace of the account, which is a much larger thing to ask for, so it has a name:

from euclid.modules.eqs import EVERY_NAMESPACE

eqs.purge_all_queues()                          # this namespace
eqs.purge_all_queues(namespace=EVERY_NAMESPACE) # all of them, deliberately

The credentials cache

login() writes ~/.euclid/credentials and reads it back on the next call, so logging in twice costs one round trip. It is the same file euclid-cli and euclid-jdk use, with the same field names, so a login from any of the three is picked up by the others. EUCLID_CREDENTIALS_FILE overrides the path, which is also how euclid hands a managed application its own credentials.

Pass use_cache(False) to force a fresh login and leave the file alone.

Signing

A login returns two credentials: a bearer token, and - when the user has one - an access key and secret. By default a session signs with the access key when it has one and presents the token otherwise, which is what AUTH_AUTO means. euclid accepts either for every action.

from euclid import AUTH_BEARER, AUTH_SIGNATURE, SigningScheme

session = Euclid.for_server(url).login("jens", "secret", auth=AUTH_BEARER)

Two signing schemes are implemented, both keyed by the same access key and secret:

Both cover a fixed set of headers rather than a set the request declares: the method, path and authority, the body digest, and the x-euclid-account-id, x-euclid-action, x-euclid-region, x-euclid-target and x-euclid-user-id headers that carry what the request is asking for and on whose behalf. euclid's server compares that list against its own for exact equality, so a signature covering more, fewer, or the same components in another order is rejected. x-euclid-namespace is not covered - a real gap rather than a simplification, and one that has to be closed on both sides at once.

Verification is implemented too, not just signing, and is what the test suite's fake gateway uses:

from euclid.auth import SignableRequest, SigningScheme

request = SignableRequest("POST", "/")
request.headers_from(incoming_headers).set_body(incoming_body).set_scheme("https")

scheme = SigningScheme.of(request)
key_id = scheme.verify(request, lookup_secret) if scheme else None

TLS

https:// URLs are verified against the system trust store. A euclid deployment usually presents its own certificate, so /etc/euclid/euclid_cert.crt is trusted alongside the system store when that file exists - the same default euclid-cli --ca-cert uses. Point ca_cert_path elsewhere, or pass verify(False) for a development server whose certificate nothing vouches for.

Errors

reason is the server's own message, pulled out of the {"error": "..."} body every euclid module answers failures with.

Retries

Two, both narrow on purpose:

• A request that failed because the connection was closed while it sat idle is sent again once, on a fresh connection. Only failures that produced no response at all qualify.
• A 401 whose body says the credentials had expired is sent again once with rebuilt headers, if rebuilding them produces something different. A wrong password or a missing permission is answered once, as before.

For a process that outlives its token, set session.token_provider to something that re-reads the credentials file; the retry then has a fresh token to use.

What EAM covers

Several of these are administrator-only server-side; session.is_admin says whether the logged-in user is one, though the server enforces it regardless. list_permissions is the exception among the role actions: it is the vocabulary rather than anybody's access, so anyone logged in may read it.

Roles and grants

A permission is the pair already on the wire — <module>:<action>, exactly what x-euclid-target and x-euclid-action carry. A role is a named set of them; a grant gives one role to one principal, scoped to an account, its namespaces and — where it matters — its resource ERNs. What a user may do is the union of the grants held by them and by every group they are in. Nothing subtracts: there are no deny rules, which is what keeps "why can they do that" a question you read rather than evaluate.

from euclid.modules.eam import ALL_NAMESPACES, BUILTIN_ROLES, EVERY_PERMISSION

session.create_role("topic-publisher", ["ens:publish-message", "ens:list-topics"])

grant = session.grant_role("topic-publisher", user.ern, namespaces=["production"],
                           resources=["ern:ens:eu-central-1:000000000000:production:topic:order-*"])

session.list_grants(principal=user.ern)      # what may they do
session.list_grants(role="topic-publisher")  # who can do this
session.list_grants()                        # everything granted in the account

session.revoke_role(grant.grant_id)

Seven roles exist without anybody creating them — BUILTIN_ROLES, from account-administrator down to transfer. The first three are computed from the permission vocabulary by rule, so a module that gains an action gains it in reader and operator on the next server build; that is why none of the seven can be created, changed or deleted. There is deliberately no administrator role: installation administration is membership of the administrator user group, which is not a grant and is not scoped to an account.

list_permissions() is the whole vocabulary, generated from what the modules actually dispatch — so a permission cannot be granted for an action that does not exist. It also names the modules that can never be granted at all (emm, emd), rather than leaving them silently missing; *:* (EVERY_PERMISSION) does not reach them either.

check_permission(user_id, target, action) is the one that earns its place. It answers allowed, the reason, and the role whose grant decided — counting the groups the user belongs to, which list_grants on the user alone does not:

answer = session.check_permission("order-service", "ens", "publish-message", namespace="production")
if not answer.allowed:
    print(answer.reason)   # e.g. no role granted here holds 'ens:publish-message'

This replaced accountGrants. User no longer carries an account_grants list, and grant_namespace_access / revoke_namespace_access are gone: a namespace grant is now a role binding scoped to that namespace, which is the same statement with the "what may they do there" part filled in rather than assumed. Listing what users may do is now one list_grants() for the account rather than a field on each user.

What ESM covers

session.esm() returns the storage client, on the session's credentials and namespace - and it follows them, so a change_namespace between two calls scopes the second one. It is the same client each time, which is what keeps an application that asks for it per operation to one connection.

upload_file splits the file into 5 MiB parts and sends four at a time by default, reading it a part at a time rather than into memory; download_file tries a single request first and falls back to fetching parts when the server says the object is too large for one. Both retry a part that failed transiently - a 5xx, or a request that got no answer - four times before giving up, because a transfer is long and made of many steps and failing one of them throws away all the others.

Attributes belong on the write rather than on a call after it. Completing an upload is finished off in the background on the server, and the object row it writes at the end carries what the upload supplied, so an attribute added in between is overwritten and silently lost:

from euclid import Variant

esm.upload_file(bucket_ern, "2026/q3.pdf", "q3.pdf",
                attributes={"tenant": "acme", "revision": 3, "checksum": Variant("binary", b"\x01\x02")},
                system_attributes={"priority": "LOW"})

The two maps are not the same one. attributes are the caller's own, listed back by list_object_attributes and meaningless to euclid; system_attributes are euclid's envelope, which travel with the object across every hop. The one euclid acts on is priority: it is what carries a producer's decision through a bucket into the queue subscribed to it. Plain Python values are tagged with the type euclid stores them under, so Variant is only needed for a tag other than the obvious one - a 32-bit int rather than a long, say.

Four actions carry an object's bytes rather than JSON - put-object, get-object, upload-part and download-part - and those present the session's bearer token rather than a signature, which is what euclid-cli and euclid-jdk do for the same four, so all three clients write objects the same way. A session created with auth=AUTH_SIGNATURE signs them anyway: it asked not to be handed a token quietly, and hashing raw bytes is exact here in a way it is not in every language.

What EQS covers

session.eqs() returns the queue client.

Receiving is a lease, not a read. A message a consumer takes is invisible to every other consumer until its visibility timeout expires, and delete_message(receipt_handle) is what says the work was done - so the delete belongs after the work, since a consumer that dies instead simply stops holding the lease and the message comes back. list_messages is the other thing: it reads a queue without touching it, and nothing it returns can be deleted by receipt handle.

receive_messages(ern, wait_time=20) is a long poll, and the waiting is the server's: it holds the request open until a message lands or the window runs out, so an idle queue costs one request rather than one per poll tick and a message comes back the instant it is sent. That request gets its own timeout - the session's is sized for an answer that comes straight back. Two cases the client handles for you: with no wait asked for, an empty queue costs no receive at all (a receive is a write); and a server with no long-poll slot free answers immediately rather than queueing behind the waiters, which is a pause and another ask rather than an empty result.

A queue's visibility, delay and size limit can be changed while it is in service, and all three apply to what is sent or received from here on:

from euclid.modules.eqs import INSTALLATION_MAX_MESSAGE_LENGTH, MAX_DELAY, MAX_VISIBILITY

eqs.set_queue_visibility(queue_ern, 120)           # seconds, 0 to MAX_VISIBILITY (43200)
eqs.set_queue_delay(queue_ern, 30)                 # seconds, 0 to MAX_DELAY (900)

limits = eqs.set_queue_max_message_length(queue_ern, 262144)
limits.max_message_length                          # what the queue holds
limits.effective_max_message_length                # what a send is measured against

Only the queue's default visibility changes: messages already in flight keep the window they were given when they were received, so this can neither expire a lease a consumer is still working on nor hold back a message its consumer has already given up on. MAX_VISIBILITY is twelve hours, the bound AWS SQS holds VisibilityTimeout to, and it is one range for both set_queue_visibility and set_message_visibility — a queue default outside what a single message may be given would be a figure no message could ever take.

A message already waiting was given its due time when it arrived, so lowering the delay does not bring it forward and raising it does not push it back — which is what keeps this from disturbing work already in the queue. MAX_DELAY is the bound AWS SQS holds DelaySeconds to and euclid keeps it: a delay smooths a burst, and anything longer is a schedule rather than a queue. A delay outside that range raises ValueError here rather than costing a round trip to be refused.

Two numbers come back from the size limit because they can differ. INSTALLATION_MAX_MESSAGE_LENGTH (zero) stores nothing of the queue's own and follows the installation's default as that changes, and the effective limit is what that default currently is — reporting the stored zero alone would read as a queue that accepts nothing. A negative limit raises ValueError. Messages already on the queue were accepted under the rule in force when they arrived, and a lowered limit is not a reason to lose them.

as_internal() returns a second view whose requests carry x-euclid-internal. Some calls observe the system rather than use it - the same get_message_count is a user's question one moment and a metric collector's poll the next - and euclid scales a module on the traffic it sees, so instrumentation that polls every few seconds would otherwise keep a pool permanently awake.

What ENS covers

session.ens() returns the topic client.

The difference from EQS is what happens to a message once it is there. A queue holds a message until a consumer takes it; a topic hands each message to every subscriber and keeps it as a record of having done so. So there is no receive and no receipt handle here - a subscriber consumes from its own queue, which is where the message was delivered:

ens.subscribe(topic_ern, eqs.get_queue_ern("orders"))
ens.publish_message(topic_ern, '{"order": 17}', priority="HIGH")

Stopping a topic holds delivery without refusing publishers. A stopped topic still accepts what is published to it and keeps it; it simply does not fan it out — which is what a subscriber being redeployed, or a downstream system taken down for the evening, actually wants. start_topic hands over the whole backlog, oldest first, as part of the call, and says how much went:

ens.stop_topic(topic_ern)                        # publishers carry on; delivery does not
...
released = ens.start_topic(topic_ern).released   # the backlog goes out here, oldest first

That backlog is real work: a topic that collected a fortnight of traffic is a fortnight of fan-out in that one call. The server pages through it and marks each message as it goes, so a start that is interrupted has delivered a prefix rather than nothing and running it again resumes. get_topic_metadata(...).held is how much is waiting, which is what says whether starting it is a moment's work.

Retention is worth setting. A topic is fanned out at publish time, so nothing ever consumes its messages and nothing else removes them — without a period the collection only grows, and every topic shares it:

from euclid.modules.ens import INSTALLATION_RETENTION, RETENTION_FOREVER

ens.set_topic_retention(topic_ern, 7 * 24 * 3600)          # a week, in seconds
ens.set_topic_retention(topic_ern, INSTALLATION_RETENTION) # 0: follow the installation's own
ens.set_topic_retention(topic_ern, RETENTION_FOREVER)      # -1: keep everything

Zero is not "keep nothing" but "whatever euclid.modules.ens.retention-period says", followed as it changes rather than frozen on the day the topic was made. RETENTION_FOREVER is not a very long period either: the server stores such a message with no expiry at all, which is what its TTL index ignores, so nothing ever removes it — the topic then grows without limit and only purge_topic empties it. The change applies to messages published afterwards; the ones already stored keep the expiry they were stamped with. A period below -1 raises ValueError here rather than costing a round trip to be refused.

A topic's size limit is not a queue's rule. set_topic_max_message_length(topic_ern, 262144) sets the largest message the topic accepts, and one number comes back rather than two — because a topic will not take a zero, so what it holds and what a publish is measured against cannot come apart the way a queue's can. Zero here is not "follow the installation's default" but a topic that accepts nothing, which the server refuses and this SDK refuses first; taking nothing for a while is what stop_topic is for, and that says so reversibly and without losing what is published meanwhile. As with retention, the change applies to what is published afterwards.

Two field names are the server's own asymmetry rather than a typo here: a message attribute travels as key throughout ENS and as name in most of EQS, and this SDK reproduces both rather than papering over either, so a request built from this documentation matches what euclid-cli sends.

What EES covers

session.ees() returns the event client — euclid's event bus, for consumers that are not euclid modules.

A subscriber registers a durable name and pulls. Nothing is pushed and no queue has to be created: receive_events claims what is waiting and ack_events deletes it, and an event claimed but never acknowledged becomes claimable again when its visibility timeout runs out — so a consumer that dies mid-work loses nothing, and the acknowledgement belongs after the work rather than before it.

The name decides fan-out. Two instances of one application share a name and compete for each event; two different applications use different names and each get their own copy.

from euclid.modules.ees import OBJECT_CREATED

ees.subscribe_events("invoice-indexer", [OBJECT_CREATED], {"prefix": "invoices/2026/"})

while True:
    for event in ees.receive_events("invoice-indexer", wait_time=20).events:
        index(event["bucketErn"], event["key"])       # an Event reads its payload like a mapping
        ees.ack_event("invoice-indexer", event.event_id)

The filter is evaluated where the event is published rather than where it is read, so a subscriber accumulates what it asked for rather than everything of that type in the installation. ESM's object events — OBJECT_CREATED, OBJECT_UPDATED, OBJECT_DELETED — carry a flat payload of strings, numbers and booleans precisely so a filter can match it by equality: {"bucketErn": ...} for one bucket, {"prefix": "invoices/2026/"} for one "directory" (a key is a path by convention only, and prefix is that convention spelled out by the server rather than by every subscriber), and {"directory": False} to skip directory markers.

An event's payload and a subscription's filter stay plain dictionaries. Their shape belongs to the event type, so parsing them would mean this SDK knowing what every module publishes — and going out of date the first time one of them adds a field.

receive_events long-polls, with a request timeout of its own; wait_time is clamped by the server to 20 seconds, and a server with no long-poll slot free answers at once with whatever is there, which a consumer loop simply asks about again.

This is the richer, server-side-filtered path. ESM's own subscribe puts a notification into a queue or topic instead — see parse_bucket_event — and the esm.subscription.* events are the plumbing behind it rather than something to subscribe to.

What EKM covers

session.ekm() returns the key client.

Key material never leaves the server: encrypt and decrypt send the bytes to the key rather than fetching the key to the bytes, and nothing in euclid.dto.ekm has a field for it. encrypt returns IV || ciphertext || tag, which is exactly what decrypt takes back.

A key is named two ways and they are not interchangeable — the server's own split, which the SDK reproduces. The ID (Key.name, what create_key returns) encrypts, decrypts and deletes; the ERN revokes, describes and tags.

Ending a key has two forms, and the difference is what happens to what it wrote. revoke_key stops it encrypting anything further and still decrypts, so it is what to reach for when the key should no longer be used but the data under it is still wanted. delete_key schedules it — seven days by default — and when that date passes, everything under it becomes unreadable: a bucket's objects, a secret's value. That window is the only chance anybody gets to notice, which is why the key is not deleted outright.

key = ekm.create_key(description="customer exports")   # AES-256 unless told otherwise
sealed = ekm.encrypt(key.name, b"account 4711")
assert ekm.decrypt(key.name, sealed) == b"account 4711"

create_certificate generates a self-signed one, for an installation that has to serve HTTPS before anybody has bought it a real one — Certificate.generated says nobody vouched for it. import_certificate takes a real one plus the private key that proves it, and the server checks the two against each other rather than letting a mismatch surface later as a handshake that fails for every caller. The private key goes in and is never handed back.

What ESS covers

session.ess() returns the secret client.

A value is encrypted with EKM before it is stored, so a secret's life is tied to a key's: deleting that key is what makes the value unrecoverable, whatever ESS still says about the secret. create_secret(..., key_ern=...) is how a set of secrets is made revocable together, and update_secret(name, key_ern=...) re-encrypts one onto another key — how a secret is moved off a key that is being retired.

Only get_secret returns a value; everything else answers with metadata alone, so a listing, a rotation or a tag change can be logged without being the thing that leaks it:

password = ess.get_secret("db-password").value
ess.rotate_secret("db-password", new_password)         # bumps version, records `rotated`

update_secret distinguishes a field being sent from a field being empty, because the server does: leaving description as None leaves the stored one alone, while passing "" clears it - and the same for value, since an empty string is a value somebody may legitimately store. An update that names none of the three raises ValueError here rather than costing a round trip to be refused.

What EKV covers

session.ekv() returns the key-value client.

A table name is unique within an account and a namespace: two namespaces may each have a suppliers table, and their items are nothing to do with each other.

A table is keyed on one attribute or on two: a partition key that identifies an item, and optionally a sort key that orders the items sharing a partition key - which is what makes a partition readable as a range. Both have declared types (STRING, NUMBER, BINARY), fixed at creation, and the type is what a comparison is made under: a NUMBER sort key orders 2, 9, 10, 100 rather than putting "10" before "9".

An item's other attributes are free-form documents - scalars, lists, nested maps - and are not declared anywhere. They are also not the tagged Variant that EQS, ENS and ESM attributes use: EKV stores what JSON can express.

from euclid.modules.ekv import GE, NUMBER

ekv.create_table("sessions", "userId", sort_key="startedAt", sort_key_type=NUMBER)
ekv.put_item("sessions", {"userId": "jens", "startedAt": 1757462400, "host": "laptop"})

for item in ekv.query("sessions", "jens", GE, 1757462400).items:
    print(item["host"])                       # an Item reads like a mapping

Three things this SDK does rather than pass straight through:

• put_item replaces rather than merges, so changing one field means reading the item, changing it and writing the whole thing back. Item therefore keeps the server's _created and _modified out of .attributes and exposes them as .created and .modified - left in, they would be written back as two attributes of the caller's own, and stick.
• get_item raises on a miss (HTTP 404), because "there is no such item" and "here is an item with nothing in it" are different. find_item is its miss-tolerant twin, returning None — dict.get to its dict[...]. Only a 404 becomes None; a malformed key still raises.
• query always sends forward, since the server reads an absent flag as descending rather than as unspecified. Asking for BETWEEN without both bounds raises ValueError here rather than costing a round trip to be refused.

query addresses a partition by key; scan reads the table. The second is fine for a small table or an export and is the wrong tool for a lookup - ScanResult.total says how much there is to get through.

What EAP covers

session.eap() returns the application client. Administrator-only server-side, all of it.

An application is deployed from an artifact already in a bucket — ESM puts it there, EAP names it — and the deployment says which buckets and queues it may reach. euclid grants those to the identity it runs as: a technical principal it creates for the application unless one is named, with no password, no login and one access key, so nothing an application leaks is a person's credential.

from euclid.modules.eap import JAVA

eap.create_application("order-service", JAVA, bucket="artifacts",
                       artifact="order-service-1.4.0.jar", queues=["orders"],
                       min_instances=2, max_instances=5)
eap.start_application("order-service")

A few things this SDK reproduces rather than smooths over:

• Deploying names things; the answer describes ERNs. A deployment takes a bucket name and an artifact key, and the Application that comes back has bucket_ern and artifact_key; the buckets and queues granted come back resolved into resources, in the namespace the application is deployed into. Names are what an operator has in hand, ERNs are what euclid stores.
• An application is identified by its namespace as well as its ID. Application.namespace says which one — two namespaces may each deploy a billing — and runtime_name is what the process, its socket and its log channel are named after, since none of those has a namespace to live in. Naming a namespace in update_application is a move: the grants are re-resolved there, and it is refused if that namespace already has an application of this ID.
• desired_state is what was asked for and state is what is running. start_application changes the first and the manager acts on it, so the application in the answer is usually still STOPPED. The two differing is an application starting up; the two differing for long is one that cannot.
• update_application sends only what it names — but buckets and queues are re-resolved together whenever either is named, so the SDK leaves both out unless you pass them. Passing either alone revokes what the other used to grant; passing both empty revokes everything, deliberately.

redeploy_application is what a new build of the same application usually wants: the artifact defaults to the one already deployed and the version to whatever the artifact's name says. A redeploy that would change neither the version nor the checksum is refused with HTTP 409 — it would restart the instances for nothing, and usually means the new artifact never reached the bucket.

What EAG covers

session.eag() returns the API gateway client. Every action here is administrator-only server-side; session.is_admin says whether the logged-in user is one, though the server enforces it regardless.

A route ID is unique within an account and a namespace rather than across the installation, so two namespaces may each publish an orders route - and namespace on a create says which one it publishes in, defaulting to the session's.

A route publishes a path prefix and says where everything beneath it goes: to an application euclid runs, or to one action of a euclid module. It is one or the other, never both and never neither - those are reached in entirely different ways, and a route naming both would leave which one wins up to the proxy. This SDK raises ValueError for that rather than costing a round trip to be refused.

from euclid.modules.eag import EUCLID_AUTH

eag.create_route("orders", "/api/orders", "order-service",
                 methods=["GET", "POST"], authentication=EUCLID_AUTH)

# The way in for something outside euclid that needs euclid itself.
eag.create_module_route("login", "/euclid/login", "eam", "login")

Module routes are what a browser needs to log in before it can call anything: without one, a front end talks to the API gateway for the application and to euclid's own gateway for its credentials - two ports, two origins, and CORS between them.

authentication is NO_AUTH (proxied as it arrives, the application enforcing whatever it requires), EUCLID_AUTH (a euclid credential — token, RFC 9421 signature or SigV4 — verified before anything is forwarded), or BASIC_AUTH (HTTP Basic against a euclid user's password, for the callers a euclid credential does not suit).

set_route_active(route_id, False) is how something stops being exposed in a hurry: the route stays exactly as it was and comes back the same, which deleting and recreating it would not guarantee. update_route changes only what it names, and namespace/region are left out of a create entirely unless you name them — the server reads an empty string as the empty namespace rather than as "unspecified", so sending one would scope the route to nothing.

list_listeners reports what the gateway was configured to serve and whether it is serving it. A listener whose port was taken, or whose certificate could not be loaded, is still listed — it is the one somebody is looking for — and serving is what says whether anything is bound. An HTTPS listener's certificate arrives flat, as a dozen certificate* fields, which the SDK gathers into Listener.certificate (or None where there is none to report):

for listener in eag.list_listeners().listeners:
    seal = listener.certificate
    print(listener.port, listener.protocol,
          "self-signed" if seal and seal.generated else "issued" if seal else "no certificate")

What ETS covers

session.ets() returns the transfer server client. Administrator-only server-side, all of it.

ETS never speaks FTP or SFTP itself. It owns the definitions — which protocol, which port, which EAM users and groups may log in, which ESM bucket the files really live in — and starting one is nothing more than writing a desired state onto a definition; euclid's manager turns that into a running process, which reads its own definition back from here.

So a file uploaded over FTP is an object in a bucket, with the events and the lifecycle every other object has. This is a protocol somebody's existing tooling already speaks, put in front of storage, rather than a second place files live.

from euclid.modules.ets import SFTP

ets.create_server("partner-drop", bucket="invoices", port=2222, protocol=SFTP,
                  user_groups=["partners"], home_directory="incoming/")
ets.start_server("partner-drop")

home_directory is the key prefix a logged-in user lands in, which is what lets one bucket serve several servers without either seeing the other's files. host_key is SFTP's, generated on first start when left empty — set it only to keep a key clients already trust. pasv_min/pasv_max are FTP's passive range, which whatever sits in front of euclid has to let through as well as the control port.

TransferServer.namespace says which namespace a server was defined in — a server_id is unique within an account and a namespace, not across the installation — and runtime_name is what its process, socket and log channel are named after, for the same reason EAP has one.

As in EAP, desired_state is what was asked for and state is what is observed, and update_server sends only what it names — a named list replaces the stored one rather than adding to it. The protocol is not updatable and so is not a parameter: which one a server speaks decides which process runs it, so changing it would be a different server.

update_server restarts a running server. The process reads its definition once, as it comes up, and nothing can tell it afterwards — so the manager applies a change by starting it again, within a few seconds. That disconnects the clients on it and loses the transfers under way; a client that retries succeeds. An update that changes nothing is still free, because the comparison is on the definition a starting process would read rather than on the fact that something was written.

Admitting a client is not the same as allowing it anything. user_ids and user_groups decide who may log in; what a client may then do — list, download, upload, rename, delete — is decided by the roles granted to it, and it is deny by default. A newly listed user can log in and do nothing:

from euclid.modules.ets import TRANSFER_PERMISSIONS, TRANSFER_ROLE

session.grant_role(TRANSFER_ROLE, group.ern, namespaces=["production"])

TRANSFER_PERMISSIONS is the seven, one per kind of command rather than per verb — so ets:list-directory covers FTP's LIST, NLST, CWD, CDUP, SIZE and MDTM as well as SFTP's OPENDIR, STAT and LSTAT. Grant fewer than all seven to narrow what a client may do. Narrowing where is by the transfer server's ERN rather than by path: a client is already confined to its home prefix, and a second path-shaped access model would be one too many to reason about.

The transfer role spans two modules on purpose — the seven ets: permissions plus esm:list-objects, esm:get-object, esm:put-object and esm:delete-object. A transfer server stores nothing of its own and every call it makes to ESM carries the client's own token, so ESM's own check applies underneath this one; a role holding only the ets: half passes the FTP check and is refused one layer down. The same goes for scoping a grant: the two halves are matched against different resources, so name both the server's ERN and the bucket's. Many clients need nothing granted specially, since reader already covers listing and downloading and operator covers everything but the deletes.

Note that none of this is session.ets()'s own administrator-only rule, and the separation is deliberate: somebody who may upload through a server must not be able to stop the server they upload through. A refused client is never told why — it gets 550 Permission denied or SSH_FX_PERMISSION_DENIED, and the reason, which names roles and grants, goes to euclid's log.

What EMO covers

session.emo() returns the monitoring client.

euclid's own modules push their samples here on their own schedule rather than being polled, because a module the autoscaler is tearing down cannot answer a poll — it simply stops pushing. An application is in the same position, and pushing puts the decision about what is worth publishing where it belongs: in the application, which is the only thing that knows. What lands here lands in the same rows EMO's own collectors write, and therefore in the same rollups, the same retention and the same graphs as CPU, memory and the module gauges.

The type is not decoration. EMO rolls five-minute rows into hourly ones and hourly into daily, and the type is what says whether that is a sum or a mean — so a counter pushed as a gauge is averaged into nonsense, and a gauge pushed as a rate is summed into more of it. Metric.rate and Metric.gauge are how that is said:

from euclid.dto.emo import Metric

emo.push_metrics("invoice-parser", [Metric.rate("invoices.parsed", 41),
                                    Metric.gauge("queue.depth", 7, {"queue": "orders"})])

list_metrics and average are administrator-only server-side: an application publishes its own numbers without special rights, and reading everybody's is a different question. A MetricQuery leaves out every field it does not narrow by, since the server reads an absent field as "do not narrow by this" and an empty string as a name that matches nothing. Note since/until rather than the wire's from/to — from is a Python keyword and could not be a field name here.

Measuring rather than pushing

push_metrics takes numbers that are already final. An application that wants to count requests and time them wants a MeterRegistry, which accumulates meters in the process and pushes them on a step — what a Micrometer registry does in euclid-jdk, and what euclid's own C++ modules do with Core::Monitoring:

with session.emo().registry("invoice-parser", common_labels={"host": hostname}) as metrics:
    parsed = metrics.counter("invoices.parsed")
    failed = metrics.counter("invoices.parsed", {"outcome": "failed"})
    duration = metrics.timer("invoice.parse")
    metrics.gauge_from("queue.depth", lambda: float(len(queue)))

    for invoice in incoming:
        with duration:
            parsed.increment() if parse(invoice) else failed.increment()

A counter and a timer report what accumulated since the last publish and start again, which is what makes them rates to EMO — the thing a rollup sums. A gauge reports what it reads at the moment of publishing and is not reset. Every registered meter is published every step, including the ones that did not move: a zero is a fact, and a gap in a graph is not.

A timer records three ways — record(seconds), with timer:, and @timer on a function — and the last two are the point: a timing written by hand at the end of a function is a timing that is not taken when the function returns early or raises. It publishes the three series euclid-jdk's Micrometer registry publishes, under the same names, so a Python application's timings graph beside a Java one's: <name>.count and <name>.total as rates and <name>.max as a gauge. Durations go in as seconds and out as milliseconds — seconds because that is what time.perf_counter and timedelta.total_seconds deal in, milliseconds because that is euclid-jdk's base unit and two SDKs reporting the same operation in different units would not be comparable. There are no percentiles, which follows from where this goes: EMO stores one value per series per interval, so a p99 would have to be computed here and pushed as a series of its own.

step=0 starts no thread at all and leaves publishing to whoever calls publish() — for an application with a loop of its own, and for a test. Otherwise a daemon thread publishes every step (a minute by default) and close(), which the with form calls, stops it and sends the step in hand.

Two things worth knowing before this is switched on in anger. Every meter is one stored row per step, forever, so the number of label combinations is the number of series — a label carrying a request id or a customer name is how a monitoring database is filled up, and the labels are decided where the meter is created because that is the one place that can. And a push that fails is counted in failed_publishes and dropped, not retried: a rate sent twice is counted twice, and a monitoring system that lies about throughput is worse than one with a gap in it. Nothing here raises at the application — a process does not stop because it could not say how it was doing — so publishes and failed_publishes are the only answer to "is the monitoring working", since a metric about pushing metrics cannot be pushed.

Writing a module of your own

euclid.ModuleClient is what the three above are built on: it knows euclid's request shape, the session's credentials and how to turn a refusal into a EuclidServiceError, and nothing else. A module this SDK has not wrapped needs a target and its actions:

from euclid import ModuleClient

class Ees(ModuleClient):
    target = "ees"

    def list_rules(self, prefix=""):
        return self.call("list-rules", {"prefix": prefix})

Ees(session).list_rules()

Set byte_actions on the subclass for any action of that module whose body is raw bytes rather than JSON; _post_bytes sends one, and those actions then authenticate the way ESM's and EKM's do.

Development

pip install -e ".[dev]"
pytest

The suite runs against a fake euclid gateway (tests/fake_gateway.py) that authenticates requests with the same rules Core::HttpActionServer::Authenticate applies, so a client that signs one thing and sends another fails there rather than in production. The SigV4 canonicalisation is additionally pinned by AWS's own published test vectors, which is what makes this SDK, euclid's C++ and euclid-jdk agree rather than merely each agree with itself.

Behind that gateway sit two stand-ins that do the work rather than stub it. tests/fake_storage.py assembles an upload's parts in part order and hands back the byte range a download asked for; tests/fake_queues.py leases messages out with receipt handles and either honours a long poll or declines it the way a server short of slots does. Both are deliberate: the part of a multipart transfer a stubbed test cannot see is exactly the part that corrupts an object, and a queue whose server always answers immediately cannot tell a client that abandons a long poll from one that does not.

Releasing

Bump version in pyproject.toml and __version__ in src/euclid/__init__.py - they have to agree, and .github/workflows/publish.yml refuses to publish if they and the tag do not - then tag and push:

git tag -a v0.3.0 -m "euclid-pdk 0.3.0"
git push origin main v0.3.0

The tag runs the tests again (the test workflow triggers on branches, so a tag push would otherwise run nothing), builds the wheel and the sdist, checks them with twine check --strict, and uploads to PyPI. workflow_dispatch does the same for whatever main says, which is what a version whose tag predates this workflow needs.

Publishing uses PyPI's trusted publishing rather than an API token: PyPI verifies the workflow's own OIDC identity, so there is no secret to rotate or leak. It has to be configured once, under the project's Publishing settings on PyPI - owner jensvogt, repository euclid-pdk, workflow publish.yml, environment pypi - and for a project that does not exist there yet, as a pending publisher.

Licence

Apache License 2.0.