aa-affiliation-refresh

Compares the corporation, alliance and faction of Alliance Auth characters with ESI - every 10 minutes for characters linked within the last two hours, every 65 minutes for every active user's main - and queues AA's own update_character for the ones that differ, so a newcomer gets the right state within about 70 minutes instead of the six-hour run_model_update cycle. These are the two lanes, recent and mains: periodic selection sweeps, each with its own schedule, priority and last_run / last_chunk markers. A user's main is also checked at login and when the main changes (debounced). When a character checked by the recent lane, by the login trigger or by the main-change trigger turns out to have changed alliance or faction while staying in its corporation, the other mains of that corporation are checked once (the cascade); the mains lane does not cascade, it covers every main anyway.

It writes nothing itself: update_character in AA does the write, the state assignment and the service hooks. It does not change states, groups or permissions, sends no notifications, keeps no history and exposes no metrics; EveCorporationInfo / EveAllianceInfo stay with run_model_update.

Requires allianceauth>=5.3,<6 on Python 3.11-3.13. No models, no migrations: all runtime state lives in the Django cache under the prefix aa_affiliation_refresh:.

Install

pip install aa-affiliation-refresh

In local.py:

INSTALLED_APPS += ["aa_affiliation_refresh"]

from celery.schedules import crontab
from datetime import timedelta

CELERYBEAT_SCHEDULE["affiliation_refresh_recent_characters"] = {
    "task": "aa_affiliation_refresh.refresh_recent_characters",
    "schedule": crontab(minute="*/10"),
    "options": {"priority": 4},
}
CELERYBEAT_SCHEDULE["affiliation_refresh_main_characters"] = {
    "task": "aa_affiliation_refresh.refresh_main_characters",
    "schedule": timedelta(minutes=65),
}

options puts the selection task of the recent lane at the priority of its chunks; without it the task runs at Celery's default 5 and waits behind AA's own tasks while its chunks would go at 4. Restart beat and the workers: beat reads the schedule at start. python manage.py check reports aa_affiliation_refresh.W001 when the refresh_recent_characters entry is missing and aa_affiliation_refresh.I001 when the refresh_main_characters entry is missing; the mains lane is optional and is disabled by deleting its entry. The key must be CELERYBEAT_SCHEDULE: AA hands that name to Celery, so an entry under CELERY_BEAT_SCHEDULE reaches neither beat nor the check.

Undo: delete the two entries and the INSTALLED_APPS line, then in the AA admin (Periodic tasks) delete or disable the two periodic tasks affiliation_refresh_recent_characters and affiliation_refresh_main_characters: beat copies CELERYBEAT_SCHEDULE into that table at start and only adds or updates rows, never deletes them, so a leftover row keeps sending a task no worker registers (Received unregistered task of type ... on every tick). Restart beat and the workers. Those two PeriodicTask rows are the only database trace; cache keys expire within an hour (the login debounce key within LOGIN_DEBOUNCE_SECONDS, a rejected mark within 86400 s) except the two last_run / last_chunk markers per lane, which have no TTL: cache.delete_pattern("aa_affiliation_refresh:*") in python manage.py shell (from django.core.cache import cache; delete_pattern is django-redis, the backend of the AA template) removes them with every other key of the module and nothing else, where cache.clear() would empty the cache of the whole installation.

Settings

Read once at import; all optional.

A value of the wrong type (a boolean counts as the wrong type for a number) or outside the range is not an error: the worker and the web process log one warning from extensions.aa_affiliation_refresh.utils at import (setting invalid name=<name> value=<value> reason=<reason> default=<default>, reason one of wrong_type, below_min, above_max; for wrong_type the value is the type) and run with the default. python manage.py check does not report this; the log at startup is the only trace.

Not settings (module constants): update_character for one character is queued at most once per 3600 s (the one-hour cache the ESI specification declares, on the single-id body); a corporation cascade runs at most once per 3600 s; 4xx answers spent on bisection are counted for the whole installation over 60 s (BISECT_ERROR_BUDGET, 30; "Limitations"); an id ESI refused alone is not asked again for 86400 s (REJECTED_TTL_SECONDS); task priorities are 4 for the recent lane, login and main change (between AA's service singles at 3 and the default 5) and 6 for the mains lane and cascades (ahead of run_model_update at 7). Bisection is what a chunk does after a 4xx other than 429: ESI refuses the whole body of POST /characters/affiliation when it contains an id it does not accept, without naming the id, so the module splits the body in two, asks for each half and repeats until a one-id body isolates the culprit, at most 2 * ceil(log2(n)) extra requests instead of n single-id ones.

The module's own single-id requests (the login and main-change triggers, --character with one id, a chunk or a bisection step of one id, the probe's [a] body) send the body [X] that update_character sends, so the next [X] may come from CCP's cache until the Expires of that answer, or for UPDATE_COOLDOWN_SECONDS (3600 s) when the answer carries none. The module records the moment per character (single_fetch:<id>, TTL to Expires or those 3600 s) and holds back an update_character that a multi-id request would queue before it: a change found by the single-id request itself is queued at once, since update_character gets the same answer. The command prints the hold as deferred until=, the same as the cooldown on last_updated_affiliations.

An id ESI refuses with a 4xx when asked alone (the bisection isolated it) is marked rejected:<id> for 86400 s (REJECTED_TTL_SECONDS), with one warning line per id, affiliation id marked rejected character_id=<id> status=<status> ttl=86400, only when another request of the same bisection succeeded: with no success at all the 4xx is systemic (a route change, a 403 for the installation) and nothing is marked. Until the mark expires the lanes and the command leave the id out of their requests and count it under rejected=; --character ID names the id and sends the request regardless. To lift a mark by hand: cache.delete("aa_affiliation_refresh:rejected:<id>") in python manage.py shell (from django.core.cache import cache), or wait for the TTL.

Why 65 minutes, not hourly

The ESI specification declares a one-hour server cache on POST /characters/affiliation (x-server-cache-ttl: 3600); the module assumes it keyed by request body; the live route on the day of the probe sends no cache header and answers X-Esi-Cache-Status: MISS to a repeated body (the result under "Cache probe"). The interval stays: without a cache it costs nothing, and CCP's documentation allows a POST route an internal cache without headers. With that cache the mains lane sends the same body on every run, so two runs closer than an hour hand the second one the cached answer: a run for nothing, and a change lands two hours late. The 65 minutes are between two queueings; between two starts it is 65 + d2 - d1, where d1 and d2 are the queue delays of the two runs. When the first run waited five minutes longer than the second (possible right after run_model_update, which starts with a jitter of up to 10 minutes) the second still hits the cache; the cost is one idle run. The recent lane is not affected: every new link changes its body, and the 120-minute window is sized for the one-hour cache. This rests on the cache being keyed by body; "Cache probe" is how to check it.

Command

python manage.py affiliation_refresh --user LOGIN [--dry-run]
python manage.py affiliation_refresh --character ID ... [--dry-run]
python manage.py affiliation_refresh --recent [--dry-run]
python manage.py affiliation_refresh --mains [--dry-run]
python manage.py affiliation_refresh --status
python manage.py affiliation_refresh --probe-cache [--character ID ID]

• --user LOGIN: every character linked to the user; no such user or no linked characters is an error.
• --character ID ...: the listed ids; ids absent from the database are printed as unknown in db: 123, 456 and dropped, no known id is an error.
• --recent: what the recent lane would select now, printed as selected 3 (window=120m): 123 (Name) created=2026-09-19T12:40:00Z, ...; selected 0 (window=120m) is a normal outcome.
• --mains: every active user's main; selected 0 is a normal outcome.
• --status: schedule entry and last-run markers per lane.
• --probe-cache: live probe of CCP's cache rules, about 70 s ("Cache probe").

Exactly one mode per call. --dry-run applies to the first four modes; --status and --probe-cache ignore it, and the probe always sends its five requests. The first four fetch ESI synchronously in chunks of CHUNK_SIZE (AA_AFFILIATION_REFRESH_CHUNK_SIZE), print esi date=... expires=... last_modified=... per chunk with at least one 200 answer (until expires the same body returns the same answer; expires=- is a header ESI did not send, and the body may be repeated at once (the live route answers a repeat with X-Esi-Cache-Status: MISS); a chunk whose ids are all marked rejected sends no request and a chunk whose ids ESI refused one by one during bisection got only 4xx: neither prints an esi line), rejected=N: 123, 456 under it when ESI refused ids with a 4xx or they carry a rejected mark ("Settings"; those ids are neither diffed nor queued), one line per changed character with the resulting state under it, then the totals. Without --dry-run the changed, non-deferred, not-yet-queued characters are queued with priority 4 and queued=M already_queued=K is printed, K counting the characters whose queued:<id> key from an earlier queueing is still set; the command never writes the database and never cascades.

Exit code 0 in every normal outcome including an empty selection; 2 with argparse's usage on stderr for an unknown option or a non-integer id; 1 with the reason on stderr for a wrong mode combination (none or more than one), an unknown user, an empty --user / --character selection, --probe-cache without --character when the database holds fewer than two mains (probe needs two character ids: pass --character A B), an ESI failure (esi request failed: status=502 cause=Bad Gateway for a 4xx or 5xx, esi request failed: RequestError: ConnectError(...) for a transport failure), in which case nothing was queued for the failed chunk and the chunks before it are already queued, and the installation being under ESI's error limit (esi error limited reset=<seconds>, - when ESI sent no reset; django-esi keeps the block until then).

--user pilot --dry-run:

esi date=Fri, 19 Sep 2026 12:00:00 GMT expires=Fri, 19 Sep 2026 13:00:00 GMT last_modified=Fri, 19 Sep 2026 11:59:30 GMT
character 90000001 (Pilot One): corporation 98000001 -> 98000002, alliance - -> 99000003
  state: Guest (unchanged)
checked=1 changed=1 deferred=0

- stands for no alliance or faction. A character whose last single-id answer is younger than an hour (AA's last_updated_affiliations or the module's own single_fetch mark, "Settings") ends with deferred until=2026-09-19T13:05:00Z and is not queued. The state line is state: Guest -> Member, state: Guest (unchanged) for a main and state: n/a (alt) for a character that is no user's main (an alt or an unlinked character). Without --dry-run the last line is followed by queued=1 already_queued=0. A one-id --character ID sends the body [ID] even with --dry-run and sets the single_fetch mark.

--status:

recent: schedule=<crontab: */10 * * * * (m/h/dM/MY/d)>
recent: last_run=2026-09-19T12:05:00Z age=7m selected=12 chunks=1
recent: last_chunk=2026-09-19T12:05:00Z age=7m checked=12 changed=1 deferred=0 queued=1
mains: schedule=datetime.timedelta(seconds=3900)
mains: last_run=never
mains: last_chunk=never

schedule=missing is the entry the check would report. last_run is written by the lane's selection task after it queued the chunks, last_chunk by each chunk when it finishes, so it shows the last chunk only: a failure in an earlier chunk of the same run is invisible here and visible only in the log. never means no marker since the cache was last emptied (cache.delete_pattern, cache.clear(), a Redis restart without persistence) or a marker the command could not read (a warning line marker unreadable printed with the output: the marker fields changed with a module upgrade; the next run rewrites it). Both markers come from tasks in the celery queue, so a worker that does not consume it ages both. Reading: last_run=never or older than two intervals - the selection task does not run: the schedule, the beat log, or the worker does not consume the celery queue; last_run fresh but last_chunk old or never while selected > 0 - the chunks fail before writing the marker (a task traceback in the worker output) or wait behind higher-priority tasks (mains chunks run at 6, behind the default 5); last_chunk fresh with checked=0 while selected > 0 - the ESI request failed (affiliation request failed in log/extensions.log) or every id of the chunk carries a rejected mark (a warning affiliation id marked rejected per id on the tick that marked it); selected=0 with complaints - "Limitations", the selection rules. A negative age= means the worker's clock is ahead of the clock of the host running the command.

Log files in the AA template: the module logs to log/extensions.log (logger extensions.aa_affiliation_refresh.*), AA to log/allianceauth.log, both at INFO and above (the module's debug lines reach only the console handler); a task traceback goes to the worker's own output (the supervisor log or docker logs), not to either file.

The admin action "Update from ESI (Celery)" on EveCharacter is the UI alternative for one character.

Runbook: a user is stuck in Guest

Run as the user that owns the AA installation, with ESI reachable.

1. In the AA admin: the user has is_active and a main character in the profile. Without a main AA gives Guest whatever the affiliations are; stop here.
2. python manage.py affiliation_refresh --status: last_run of the recent lane older than two intervals of the printed schedule=, or never - the selection task does not run: check the schedule, the beat log and whether the worker consumes the celery queue; last_run fresh but last_chunk old while selected > 0 - the chunks fail, search the worker output for a traceback of aa_affiliation_refresh.refresh_affiliations. In both cases stop here.
3. python manage.py affiliation_refresh --recent --dry-run: the newcomer is not in selected - outside the window or without an OwnershipRecord ("Limitations"); go to step 4.
4. python manage.py affiliation_refresh --user LOGIN --dry-run.
5. No difference printed - check the character's last_updated_affiliations and the states in the admin (member_corporations / member_alliances, priorities); when ESI returns the old corporation, repeat after the expires printed by the command, at once with expires=-.
6. A difference with deferred until= - wait until until (CCP's cache on this character); the lane queues the update itself.
7. state: Guest (unchanged) - the corporation is in no state; fix the states in the admin, the module is not involved.
8. state: Guest -> Member - repeat without --dry-run; the last line is queued=1 already_queued=0. The update_character task runs at priority 4, in a minute on an idle worker; wait for Updating <user> state to Member in log/allianceauth.log (written by AA when it saves the character). The module writes nothing at this step: its affiliation changed line comes from the lanes, not from the command.
9. queued=1 but no Updating <user> state to within five minutes - update_character failed or the worker does not consume the celery queue: search the worker output for a traceback of allianceauth.eveonline.tasks.update_character; the usual cause is HTTPNotModified ("Known AA and django-esi defects"): run the admin action "Update from ESI (Celery)" on any other character, then repeat step 8. After a failed update_character the key queued:<id> stays for up to 3600 s (UPDATE_COOLDOWN_SECONDS, not a setting) and a repeat of step 8 prints queued=0 already_queued=1: queue again after the key expires, or first run cache.delete("aa_affiliation_refresh:queued:<id>") in python manage.py shell (from django.core.cache import cache). No traceback and no Updating line - the worker does not consume the queue; the beat and worker logs are the next place.

Limitations

• Re-linking a character that the same user already linked before creates no OwnershipRecord (AA skips the duplicate by owner_hash and character): the recent lane never sees it; it is checked by the mains lane if it is a main, otherwise by run_model_update.
• OwnershipRecord.created is auto_now, not auto_now_add: saving a record in the AA admin moves its date and pulls the character into the recent lane for one more check. AA itself never re-saves records.
• Characters in Doomheim (corporation_id 1000001) are excluded from both lanes, as in AA, and by the login and main-change triggers. A character ESI leaves out of its answer is skipped without a log line; run_model_update picks it up.
• A user with is_active=False (email not confirmed) is refreshed by the recent lane and gets a state by corporation; State.objects.get_for_user in AA ignores is_active, run_model_update behaves the same.
• The ESI daily downtime (11:00 UTC) costs one warning line affiliation request failed characters=<n> error=<class> status=<status> cause=<text> per chunk of a run that falls into it (the mains lane runs ceil(mains / CHUNK_SIZE) chunks); status= is the HTTP status of a 4xx or 5xx and - otherwise, cause= ESI's error text for those and the transport exception for the rest; the next tick retries, nothing to do.
• ESI error budget (100 errors per minute per installation, shared with every AA task and plugin): the module produces at most one 4xx per chunk per tick, and with bisection about 18 extra requests, 10 of them 4xx, per rejected id. The 4xx answers spent on bisection are counted in the cache for the whole installation over 60 s (BISECT_ERROR_BUDGET, 30: three rejected ids per minute); the chunk whose 4xx exceeds the budget fails as a whole with affiliation request failed. The rejected ids isolated in that call, the one whose 4xx exceeded the budget included, are marked (rejected:<id>, "Settings") and the next tick isolates the next three or four (four when the budget runs out on a single-id body), so a chunk with any number of rejected ids completes in at most ceil(n / 3) ticks; a systemic 4xx (a route or version change, a 403 for the installation) marks nothing and the chunk is retried whole, at most 31 4xx per minute plus one per further chunk, never 2n - 1 per chunk. The module's window starts at the first 4xx and is not aligned with ESI's minute (X-ESI-Error-Limit-Reset), so two module windows can fall into one ESI minute and the bound against ESI is doubled, about 2 x (31 + chunks - 1); at the lanes' cadence it stays below 100. Past 100 errors ESI answers 420 and django-esi blocks every ESI request of the installation until the window resets; a 420 is caught by django-esi before the request and retried by rate_limit_retry_task.
• The main-change receiver on UserProfile fires on every full save() without update_fields, which in AA 5.3.1 is a change of language, theme or night mode, the registration, the re-attach of a returning character in the SSO backend, the sidebar toggle (minimize_sidebar) and a save in the admin: each costs one ESI request for the main, at most once per LOGIN_DEBOUNCE_SECONDS per character.
• The module needs cache errors to raise. With DJANGO_REDIS_IGNORE_EXCEPTIONS = True (not set by the AA template) django-redis returns None instead: a claim of a queued, cascade, login or bisect_errors key then raises CacheUnavailableError with the text cache returned None (DJANGO_REDIS_IGNORE_EXCEPTIONS?) key=<key> (the task fails with a traceback, the login trigger logs one error line main character refresh failed ... error=CacheUnavailableError cause=cache returned None (...) without a traceback and the login goes through), while the lane markers are written without a check and --status prints never for lanes that run.
• update_character that fails inside AA after the ESI request went through (get_or_create_esi of the corporation or alliance) leaves the ETag stored: the next queueing an hour later gets a 304, HTTPNotModified, and fails again until AA's update_character for another character overwrites the shared ETag key; the module's own requests store no ETag, so its lanes never clear it, and without an operator the clearing waits for the next run_model_update (six hours at most). The module does not fix this; runbook step 9 names the action.
• python manage.py check reads settings.CELERYBEAT_SCHEDULE, not the django_celery_beat table beat fills from it (Undo in "Install"): an entry disabled by hand in the admin is not seen. --status shows it as a stale last_run.
• Emptying the cache (cache.clear(), Redis without persistence) resets the markers to never and drops the queued, cascade, single_fetch and debounce keys: one duplicate update_character or cascade may follow; a duplicate update_character fails with HTTPNotModified (a traceback in the worker log) and changes nothing. When the broker refuses a queueing, the login, queued or cascade key claimed for it is released and the next tick retries.

Cache probe

python manage.py affiliation_refresh --probe-cache

Live ESI, about 70 s: the bodies [a, b], [b, a], [a], [a, b] with a 60 s pause after the first, a and b being the two lowest main character ids (--character A B overrides), then the body [2147483000], an id inside int32 that is not allocated. The verdict comes from X-Esi-Cache-Status of the four answers when ESI sends it on each of them, otherwise from Expires, Last-Modified and ETag, never Date: a CDN may stamp the current time on a cached answer. Valid when nobody sent these bodies within the previous hour (from this application, if ESI keys its cache per application; the scope of that key is not verified): a warm cache answers the first [a, b] with a HIT, and the verdict leans towards a wider cache. Output: one line per body, ab [90000001, 90000002] expires=... last_modified=... etag=... cache_status=MISS, - for a header ESI did not send, then cache=<verdict> bad_id=<outcome>.

bad_id=rejected: a 4xx for the whole body, the bisection path is what runs in production. bad_id=skipped: ESI answers without the id, the 4xx path is untested on live ESI and bisection stays as insurance. bad_id=returned: the id 2147483000 is allocated and the outcome says nothing about the 4xx path; raise PROBE_MISSING_CHARACTER_ID in probe.py, there is no command-line switch for it. bad_id=error:<class> (error:HTTPServerError, error:ReadTimeout, error:HTTPClientError for a 429): the fifth request failed with a 5xx, a 429 (ESI's bucket limit, not a verdict on the id) or a transport error after the four bodies were answered; cache= stands, the 4xx path is untested, rerun for it.

Result (2026-09-21, statuses read with curl): no cache headers on any body, X-Esi-Cache-Status: MISS on every body including the repeat, bad_id=rejected; a later run is recorded here as Result (date): cache=<verdict> bad_id=<verdict>. Re-run before changing the mains interval and after any CCP announcement about caching on this endpoint; the table above says what to change.

Known AA and django-esi defects

• EveCharacter.update_character() does not catch HTTPNotModified: a repeat of the same single-id body without another affiliation request in between gets a 304 and a traceback. Triggers: the admin action "Update from ESI (Browser)" twice in a row, two update_character(X) tasks in a row, a retry after a failure in get_or_create_esi. The module avoids it with the one-hour deferral and the queued key; the fix is an upstream MR.
• django-esi 9.10.0 keys its response cache and its ETag without the request body (_extract_body_param() runs before _cache_key() in esi/openapi_clients.py): all PostCharactersAffiliation calls share one key. AA works only because every AA call passes store_cache=False; a third-party app calling result() with the default store_cache=True hands update_character another body's answer, and it takes affiliation[0] without checking character_id. The module passes use_cache=False and matches by character_id; reports go to django-esi (key with body) and AA (character_id check).

CI

.gitlab-ci.yml runs on merge requests, on pushes to main and develop, and on tags; a push to a branch with an open merge request starts no second pipeline, a branch without one runs nothing. Every job is one nox session in the ghcr.io/astral-sh/uv:python3.13-trixie image and is reproduced locally with uv run nox -s <session> (the tests job needs uv python install 3.11 and so on first).

Release: push a protected tag vX.Y.Z. build strips the v and requires X.Y.Z to equal project.version in pyproject.toml and to have a ## [X.Y.Z] section in CHANGELOG.md (that section is the release notes); either mismatch is red before anything is published. publish uploads dist/*.whl and dist/*.tar.gz to PyPI by Trusted Publishing: no token is stored in CI variables, the publisher is configured on PyPI (namespace eveo7, project aa-affiliation-refresh, pipeline file .gitlab-ci.yml, environment pypi, compared case-sensitively), and an unprotected tag is refused by the job rule. While the project does not exist on PyPI, the publisher is added as a pending publisher on the account page "Publishing" (https://pypi.org/manage/account/publishing/); the first upload creates the project and turns it into an ordinary publisher. release creates the GitLab release from the notes with a link to the PyPI page. A red publish leaves nothing on PyPI; a version that reached PyPI cannot be uploaded again (PyPI refuses a file name it has seen), so a broken release is followed by the next version, not by a re-tag.

Locally, gitlab-ci-local <job> (docker required) runs a job in the same image; gitlab-ci-local --list shows the jobs of a push pipeline, tag jobs appear with --variable CI_COMMIT_TAG=v0.1.0 --variable CI_COMMIT_REF_PROTECTED=true, the scheduled job with --variable CI_PIPELINE_SOURCE=schedule. Runner tags are ignored locally; publish and release cannot run locally (no OIDC token, no GitLab API).

Russian: README.ru.md.