aa-discord-relay

Mirror selected Discord channels into EVE Online in-game mailing lists and into Telegram chats.

An Alliance Auth module. See docs/ for design and implementation plans.

What it does

A Discord channel is watched by a gateway bot. Every message posted there is recorded and delivered to each destination configured for that channel. The main case is a fleet ping posted by aa-fleetpings through a webhook: it reaches the people who read in-game mail and the people who read Telegram, without anyone reposting it by hand.

One channel can have several destinations, and each pairing carries its own templates, hourly quota, maximum age and footer language.

Requirements

• Alliance Auth 5.x.
• Redis, reached through a django-redis cache backend shared between processes. The resource lock that serialises each send requires Redis specifically: on any other backend - Memcached included - every acquire raises and each delivery retries until it expires, unsent. The send throttle and the bot heartbeat are cross-process cache state as well, and on a per-process backend they silently do nothing. A Django system check warns when the default cache is not django-redis.
• Celery workers and celerybeat.
• A process supervisor for the gateway bot (discordrelay_bot).
• A Discord application with the privileged MESSAGE_CONTENT intent enabled. Without it messages arrive empty and every one of them looks like it has nothing to mirror.

Concepts

Four entities, and one mirror needs two screens because of it:

Entity	What it is
Channel subscription	a Discord channel that is read at all
Route	one channel-to-destination pairing: templates, quota, age, language
Destination	where messages go: an EVE mail address set, or a Telegram chat
Delivery	one attempt to put one message into one destination

A message accepted from Discord produces one delivery per enabled route. Deliveries are independent: a broken Telegram chat does not stop the mailing list, and a resend acts on one delivery, never on the whole fan-out.

A channel can be narrowed to named sources with an allow-list, edited where the channel is. Left empty it mirrors every source, as before; listing a webhook or an author makes the channel relay only the sources on the list. This mirrors just aa-fleetpings out of a busy channel, and keeps a reverse relay's own posts from being mirrored back.

Messages can be tagged from their text. Tag rules map a regular expression to a label; a route template's {tags} placeholder — in the subject or the body — expands to the labels of every rule whose pattern is found in the message, joined by a separator set once in the global settings. Rules are global, so every mirrored channel is tagged the same way, and a body without {tags} is unaffected.

A route template may also carry {event_time}: the time the message announces, shown in EVE time as YYYY.MM.DD HH:MM EVE (the same format as {date} and the footer). An explicit Discord <t:...> timestamp is taken as written; otherwise the text is searched, in Russian and English, for a clock time next to an optional date ("завтра в 19:30", "18:00 on 26.07", "через 2 часа"). Only results between the ping and thirty days ahead are trusted; anything else, and any message naming no time, renders as empty text.

The default subject template is [{channel}] {embed_title}, where {embed_title} expands to the title of the message's first Discord embed — the headline aa-fleetpings puts on a ping — and to empty text for a plain message that carries no embed.

Four more placeholders name where a message came from: {author_nick} is the author's server nickname (on an Auth-managed server usually the main character's name), {author_top_role} their highest guild role (empty for members with no role), {guild} the server name, and {jump_url} a link back to the original Discord message. A webhook message shows the same webhook: #channel label in {author_nick} as in {author}, and carries no role.

Installation

1. Install the package into the Auth virtualenv and add discordrelay to INSTALLED_APPS.
2. Run python manage.py migrate.
3. Configure Alliance Auth's Discord service, including its DISCORD_BOT_TOKEN, and enable the MESSAGE_CONTENT intent for that bot.
4. Invite the bot and grant View Channel only on the channels that are mirrored. The bot reads everything it can see; the permission is the boundary.
5. Announce in each mirrored channel that it is mirrored. People who post there are being republished to a wider audience.
6. Register the sender character: open EVE mail destinations in the admin and use Register sender character. Use a dedicated service character, not a personal one - every mail lands in its Sent Items and replies go to it.
7. Import the mailing lists the sender is subscribed to: Recipients → Import mailing lists. The sender must be a member of every list it mirrors into.
8. For Telegram, add a bot with its token under Telegram bots, add the bot to the chat, and create a Telegram destination naming the chat (and a forum topic, if any).
9. Create a Channel subscription for the Discord channel and add one route per destination inline.
10. Start the gateway bot under the supervisor and run python manage.py discordrelay_check.

Upgrading

supervisorctl stop discordrelay_bot
python manage.py migrate
supervisorctl restart worker
supervisorctl start discordrelay_bot
python manage.py discordrelay_check

Periodic tasks

Both schedules belong in the Auth installation's local.py:

CELERYBEAT_SCHEDULE["discordrelay_purge_old_history"] = {
    "task": "discordrelay.tasks.purge_old_history",
    "schedule": crontab(minute=0, hour=3),
}

# Not optional. requeue_stalled is the only thing that recovers a row
# claimed by a worker that died, and the only thing that queues a row
# whose on_commit callback never reached the broker: without it those
# rows sit forever, never retried, never reported, and still counted
# against both hourly quotas.
CELERYBEAT_SCHEDULE["discordrelay_requeue_stalled"] = {
    "task": "discordrelay.tasks.requeue_stalled",
    "schedule": crontab(minute="*/5"),
}

The five-minute interval is not a hard requirement, but it must stay at or below the shortest route max_age: an orphaned pending row is only picked up roughly ORPHANED_PENDING_AFTER_SECONDS (120s) after it was created, plus up to one beat interval on top - run it less often and a lost task can wait longer than a fleet ping lives.

Dedicated task queue

By default every relay task goes to Celery's default queue (celery), shared with all of Alliance Auth. A backlog there - thousands of allianceauth.eveonline.tasks.update_corp tasks after a corporation refresh - holds deliveries in pending for as long as it takes to drain, and requeue_stalled waits in the same line, so recovery stalls too.

DISCORDRELAY_TASK_QUEUE in local.py sends all three relay tasks (deliver_message, requeue_stalled, purge_old_history) to a queue of their own: fan-out, reschedules, the admin resend action and beat all use it. A retry is the exception: Celery publishes it with the exchange and routing key of the attempt it repeats, so it returns to the queue that attempt came from, whatever the setting says. A CELERY_TASK_ROUTES entry does not work for this: Auth's generated celery.py assigns app.conf.task_routes after it reads Django settings, overwriting it.

Value	Effect
absent or None (default)	Celery's default queue
a non-empty string	that queue; a worker must consume it
empty string, whitespace, non-string	ImproperlyConfigured whenever a relay task is queued

A queue no worker consumes fails silently. Nothing raises and nothing is logged: deliveries wait in the broker unread until they expire, and requeue_stalled waits beside them. discordrelay_check reports this as [FAIL] task queue.

The setting must be identical on every process that queues relay tasks - every web process, beat, every worker, and every replica of each. Workers queue relay tasks too: reschedules, and the deliveries requeue_stalled puts back. A process without the setting keeps sending to the default queue; nothing is lost there, but those tasks wait behind the shared backlog again.

To switch, in this order:

1. Add a worker for the queue to the supervisor, next to the existing ones, and start it:

   celery -A myauth worker --pool threads --concurrency 2 -Q discordrelay
   

2. Set the queue in local.py:

   DISCORDRELAY_TASK_QUEUE = "discordrelay"
   

3. Restart every process that queues relay tasks: the Celery workers, beat, the web server (the admin resend action) and discordrelay_bot (fan-out).

4. Run python manage.py discordrelay_check. Expected line: [OK] task queue: discordrelay, consumed by <worker name>.

To undo, remove the setting and restart the same processes. Tasks already in the dedicated queue are not moved back, and retries of attempts taken from it keep arriving there. Then either:

• keep the dedicated worker running until it holds nothing: celery -A myauth inspect scheduled and celery -A myauth inspect reserved list no relay task for it, and redis-cli -n <broker db> --scan --pattern 'discordrelay*' prints no key. LLEN discordrelay alone reads 0 too early: with Auth's priority_steps a message of priority 5 lives under the key discordrelay\x06\x165;
• or stop it at once. Nothing is lost: requeue_stalled, now on the default queue, re-queues a pending row older than 120 seconds once its retry time has passed, and a sending row claimed more than 240 seconds ago. Such a delivery is late by up to one beat interval, and is expired instead if that takes it past the route's maximum age. Messages left in the old queue stay in the broker; if a worker ever consumes them, they are handled like any redelivered task: the row's state decides, and a row already sent is left alone.

The threads pool enforces no Celery time limit (Celery 5.6.3); the relay tasks set none either. A send is bounded by its HTTP timeouts instead: 20 seconds per Telegram request (TIMEOUT_SECONDS in discordrelay/services/telegram_api.py), and for ESI django-esi's ESI_REQUESTS_CONNECT_TIMEOUT / READ_TIMEOUT / WRITE_TIMEOUT / POOL_TIMEOUT (5 / 10 / 5 / 5 seconds by default in django-esi 10.0.0) with up to ESI_SERVER_ERROR_MAX_RETRIES (3) retries on server errors. An installation-wide CELERY_TASK_TIME_LIMIT is not applied on this worker.

[FAIL] task queue means at least one worker replied and none of them consumes the queue. [WARN] task queue means no worker replied within 3 seconds, so the check could not tell. The check sees only workers running at that moment; a worker that stops later is not reported.

Tag rules

Tag rules label a message from its text: each rule pairs a regular expression with a label, and a route template's {tags} placeholder expands to the labels of every matching rule (see Concepts). Rules are global and edited under Tag rules in the admin.

A starter set tuned against real fleet pings ships with the module but is not installed automatically - a deployment that tags nothing needs no rules. Install it once:

python manage.py discordrelay_seed_tags            # create the starter set
python manage.py discordrelay_seed_tags --dry-run  # show what it would do
python manage.py discordrelay_seed_tags --update   # pull shipped patterns
                                                   # back over existing rules

The command is idempotent and keyed by label: run it twice and the second run adds nothing, a rule you edited is left alone unless --update is given, and a rule you disabled stays disabled.

The starter labels, matched in Russian and English, case-insensitive:

Label	Fires on
#important	важно/важен/срочно/макс/асап · important/urgent/max/asap
#cta	CTA / КТА, the call to arms
#ops	опс · ops
#roam	роум/роам · roam/roaming
#defense	защита/деф/оборона · defend/defense/hold
#battle	бой/атака/запрыг/замес/ганк/пвп · fight/battle/brawl/attack/gank/pvp
#capitals	капы/дреды/кэрриеры/супера/титаны/факсы · cap/dread/carrier/titan/fax
#formup	сбор/вылет/логин/инвайт/плюсуем/джойн · form up/log in/undock/invite/join
#keepstar #fortizar #astrahus #raitaru #azbel #sotiyo #athanor #tatara #metenox #ihub #poco	the structure named in the text, one tag per type
#timer	структура/таймер/цитадель/реинфорс/рефа · structure/timer/citadel/reinforce, named no more precisely
#entosis	энтоз/тозер/тостер · entosis, the sovereignty capture

Several structure tags also answer to fleet slang beyond the type name: #fortizar to «фортик», #sotiyo to «сотейка», #ihub to «хаб/клайм/суверенитет» (the sovereignty fight), #poco to «кастом/таможня» (the customs office).

The patterns are anchored on word boundaries so a short acronym does not fire inside an ordinary word ("cta" does not match "контакта"). Edit or extend them freely; to try a pattern before it goes live, use Preview tags on the Tag rules page - paste a real ping and it reports the labels that message would earn, through the same matching the delivery path uses.

When a template uses {tags} but no rule is enabled, every message renders the placeholder as nothing. discordrelay_check reports that as a warning, and python manage.py migrate prints a hint pointing at the seed command.

Diagnostics

python manage.py discordrelay_check
python manage.py discordrelay_check --route-id 3
python manage.py discordrelay_check --route-id 3 --send-test-message

The command walks the chain in order - cache backend, beat schedule, task queue (see Dedicated task queue), reverse relay, bot heartbeat, channels without routes, tag rules, then each route's destination - and exits non-zero when anything failed. --send-test-message needs a route id: one real message per route at once trips the remote spam limit and spends the installation-wide ESI error budget.

Delivery states:

State	Meaning
pending	accepted, waiting for a worker
sending	a worker holds it right now
sent	the remote side accepted it
failed	terminal failure; the reason is on the row
expired	older than the route's maximum age, never sent
dry_run	rendered and recorded, deliberately not sent
skipped	rendered to empty content, so nothing was sent

The gap alarm depends on the cache. How long the bot was away is measured from the heartbeat, which lives in the shared cache. If that entry is evicted - an allkeys-lru Redis under memory pressure, or a cache flush - the outage reads as "never seen" and no gap notification is sent. The catch-up itself is unaffected; only the announcement is.

Channel visibility cannot be checked from here. A bot without View Channel behaves exactly like a bot with no subscription: no error, no message, nothing in the history. If a channel is silent and every check passes, verify the permission in Discord.

Duplicate policy

A worker can die between handing a message to the remote side and recording that it did. What happens next differs by transport, and the difference is not a bug:

• EVE mail is recovered. The sender's Sent Items are searched for the message before it is sent again, so a crash costs a delay, not a duplicate. This needs the esi-mail.read_mail.v1 grant; without it the check cannot run.
• Telegram is not recovered. The Bot API cannot be asked what it has already sent, so a crash mid-send leaves a duplicate message in the chat. The admin says so on every Telegram destination, and discordrelay_check reports it as a warning.

Rules and limits

• The EVE Online Developer License Agreement applies. §2.3 (no automation that plays the game for a user) and §2.5 (respect the ESI error limit) are the two that bound this module.
• ESI_SSO_CLIENT_ID is shared by the whole Auth installation. The ESI error budget is shared with it: a retry storm here returns 420 on every ESI route for every other module.
• Two quotas apply to every send: the route's own hourly limit, and the transport resource limit on the sender character or the chat. Per-route quotas add up - five channels at twenty an hour on one character is a hundred, against an in-game limit of roughly five a minute.
• Use a dedicated service character as the sender.
• The Telegram Bot API limits a group to roughly twenty messages a minute; the per-bot and per-chat spacings are configured separately.
• {tags} is cosmetic, not moderation or routing. Rules are matched on the message text without attachments and before truncation, and the separator is inserted as data, so markup or line breaks in it stay literal. A rule pattern is run against untrusted Discord text under a time limit, so a pathological pattern is dropped rather than allowed to stall a delivery.
• {event_time} is best-effort cosmetics, the same as {tags}: a misread or missing time renders as empty text and never fails a delivery. EVE-time markers after a clock reading ("19:30 ET", "21:00 евы") are read as UTC, and relative phrases count from the moment the message was posted, so a retried delivery resolves the same instant.

Secrets at rest

The Telegram bot token is stored in the database so the worker can send with it. The admin never puts it on a page - it is absent from every list and the edit form renders a blank field, not the stored value - but the row itself holds it in the clear, the same way Alliance Auth stores the ESI refresh tokens that back every EVE mail send. Encrypting this one field alone would not lower the exposure while those tokens sit beside it in plaintext, so protect them together at the storage layer:

• Encrypt the database volume, or use a managed database that encrypts at rest, so a lost or discarded disk does not surrender the tokens.
• Restrict who can read a database dump, and treat backups as secret material: a dump is a full copy of every token above.
• If a disk or a dump is ever exposed, rotate the secrets. A Telegram token is reissued from BotFather and pasted over the old one in the admin; the ESI tokens are re-granted through the normal SSO flow.

Development

Run every finite local gate, including the wheel and MariaDB suite:

uv run nox -s all

Preview dependency upgrades without changing uv.lock with uv run nox -s deps_upgrade; apply them with uv run nox -s deps_upgrade -- --apply.

To boot a persistent local demo, export ESI_SSO_CLIENT_ID, ESI_SSO_CLIENT_SECRET, ESI_USER_CONTACT_EMAIL, and Alliance Auth's DISCORD_BOT_TOKEN, then run:

uv run nox -s demo

Register http://localhost:8000/sso/callback/ as the callback. The session sets that URL itself instead of inheriting the production callback, validates the remaining settings, and starts a Docker Compose stack with MariaDB, Redis, the Django server, Celery worker and beat, and the Discord gateway bot. Ctrl+C removes the containers; the MariaDB volume is retained for faster subsequent starts.

Not supported in this version

• Mail to individual characters: it needs CSPA charge handling.
• Detection of a Discord bridge reposting into a mirrored channel.
• Drift detection for the Telegram Bot API, unlike the ESI contract session.