reach_commons
Reach's shared Python utility library — SMS encoding, phone validation, structured logging, persistence wrappers (Mongo, DynamoDB, S3, SQS, Firehose, KMS), Redis rate-limiting, and HTTP clients for internal Reach services (event-processor, callback-processor, reach-ops, reach-data-bridge), plus third-party integrations (Outscraper, Bandwidth).
Distributed as the reach-commons wheel on
PyPI. Source of truth: https://gitlab.com/reach.ai/reach-commons.
Install
pip install reach-commons
Pin a version in requirements.txt / pyproject.toml like any other dep. Releases
are published from main by GitLab CI (see Release flow).
Quick example
from reach_commons.sms_smart_encoding import MessageSmartEncoding
msg = MessageSmartEncoding("Hello world!")
print(msg.encoding, msg.length, msg.segments) # GSM-7 12 1
Repo layout
reach_commons/
├── app_logging/ structured logger, http logger, deprecation tracker
├── clients/ thin HTTP wrappers for Reach internal services + 3rd-party APIs
│ ├── _auth.py shared Bearer-header helper (reads lambda_function_api_key)
│ ├── callback_processor.py
│ ├── event_processor.py
│ ├── nightly_lambda.py
│ ├── outscraper.py
│ ├── reach_data_bridge.py
│ └── reach_ops_api.py
├── credentials.py one named accessor per Parameter Store credential
├── mongo/ MongoDB customer persistence (sync + async)
├── reach_aws/ boto3 wrappers (dynamodb, s3, sqs, firehose, kms, rate limiter)
├── reach_base_model.py
├── redis_manager.py
├── sms_smart_encoding.py GSM-7 / UCS-2 detection + segment math
├── utils.py
└── validations.py phone-number validator (Twilio Lookups)
tests/ pytest, every suite uses @pytest.mark.parametrize
Security model
Starting with version 0.18.57, the wheel published to PyPI contains zero
credentials. Every secret consumed by reach_commons is fetched at runtime from
AWS Systems Manager Parameter Store (SecureString), sourced from
/terraform/shared-config.tf (shared_ssm_secure).
Why this matters: reach_commons is published to public PyPI. Anyone can
pip download reach-commons. The wheel is intended to be safe in that posture
— if it gets shared outside Reach (forked, cloned by an external dev, included
in a third-party tool), no Reach credentials travel with it.
Access control lives in IAM: a consumer Lambda can only read the secrets
its execution role grants ssm:GetParameter on. No hardcoded fallbacks, no
setattr side-channels. The one environment override is the internal API bearer
(below), which is unset in every Reach deployment.
Credential accessors
Every credential kept in Parameter Store has one named accessor in
reach_commons/credentials.py, and that module is the only place its path is written.
A value shared by several services is one parameter named after its vendor
(/stripe/{env}/secret-key), never after a consumer, so rotating it is one write.
from reach_commons import credentials
credentials.preload(credentials.stripe_secret_key, credentials.sendgrid_api_key)
def handler(event, context):
stripe.api_key = credentials.stripe_secret_key()
preload(...) at module import resolves the credentials during the cold start in one
batched GetParameters call (ten names per call), so a missing IAM grant or parameter
raises there instead of on the first request, and a deploy that starts many containers
at once does not exhaust Parameter Store's request rate. Handlers
then call the accessor itself, which reads the in-process cache and picks up a rotation
within the TTL. Do not copy a value into a module-level constant: that pins it for the
life of the container.
{env} resolves to staging / prod from the ENV Lambda env var (defaults to
Staging). The commons clients read theirs the same way: _auth uses
reach_internal_api_key, CommonsPhoneNumberValidation the Twilio pair, and
OutscraperClient the Outscraper token.
Required consumer Lambda IAM
Grant ssm:GetParameter on each exact parameter the consumer reads, in its own
environment only. Never a prefix and never GetParametersByPath: a by-path read is how
one service once pulled the other environment's secrets into its state.
statement {
sid = "ReadOwnCredentials"
effect = "Allow"
actions = ["ssm:GetParameter", "ssm:GetParameters"]
resources = [
"arn:aws:ssm:us-east-1:383836045505:parameter/api-keys/${lower(var.tag)}/lambda-function-api-key",
"arn:aws:ssm:us-east-1:383836045505:parameter/stripe/${lower(var.tag)}/secret-key",
]
}
The parameters use the default alias/aws/ssm key. /terraform/ssm-credential-reads.tf
is the reference for the lambda roles in that repo.
Caching and rotation
reach_commons._ssm caches each SSM value in-memory with a 1-hour TTL per
process. Cold start = 1 GetParameter per credential (~50ms each); warm
container reuses the cached value until expiry.
After 1h, a rotation in SSM is picked up automatically on the next call — no redeploy needed. To force an immediate refresh, redeploy / recycle the Lambda containers.
Test / local override
Test suites replace Parameter Store before importing code that preloads credentials:
import reach_commons.credentials
reach_commons.credentials.override_for_tests(lambda path: f"test-value-for:{path}")
Each affected client accepts explicit kwargs so tests and local tooling can bypass SSM entirely:
CommonsPhoneNumberValidation(account_sid="ACtest", auth_token="...")
OutscraperClient(token="...")
reach_api_bearer_header() has no kwarg path — tests should
monkeypatch.setattr(reach_commons.clients._auth, "get_secure", lambda p: "...").
Internal API bearer from the environment
reach_api_bearer_header() uses REACH_INTERNAL_API_KEY as the token when that
variable is set and non-blank, and reads /api-keys/{env}/lambda-function-api-key
from Parameter Store otherwise. Reach sets it nowhere, so Reach keeps the SSM read; a
deployment outside Reach AWS (a fork of the voice relay or an agent) sets it instead of
granting SSM access.
Voice contract
reach_commons/schemas/voice_agent.py is the relay <-> agent voice contract, versioned
by VOICE_CONTRACT_VERSION (currently "1"), which every model carries as
contract_version:
| Model | Direction | Route |
|---|---|---|
VoiceSetupRequest / VoiceSetup |
relay -> agent, answered synchronously | POST {agent}/voice/setup |
VoiceAgentRequest |
relay -> agent, fire-and-forget | POST {agent}/voice/agent-request |
VoiceAgentCallback |
agent -> relay, fire-and-forget | POST {relay}/reply |
Unknown fields are ignored and a missing contract_version reads as the current one,
so adding an optional field does not bump the version; a change an older peer cannot
ignore does. business_id is an int for a numeric tenant (a numeric string is read as
one, so that wire is unchanged) and a string for any other tenant id.
Local development
poetry install # runtime + dev deps into .venv
poetry run pre-commit install # one-time: enables commit/push hooks
poetry run pytest # full suite
poetry run ruff check reach_commons tests
poetry run ruff format --check reach_commons tests
poetry run pyright reach_commons tests
Python floor: ^3.8. CI runs on python:3.13-slim.
Pre-commit hooks
.pre-commit-config.yaml installs three layers:
- On commit (fast): trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-merge-conflict, check-added-large-files, ruff (lint + autofix), ruff-format.
- On push (slower): pyright with the runtime deps as
additional_dependencies.
Pyright runs only on push so day-to-day commits stay fast.
Release flow
GitLab CI publishes to PyPI automatically. Workflow:
- Open an MR against
main. - CI runs
lint+typecheck+teston every push (verify stage). - Merge to
main. - The
publish_pypijob compares theversioninpyproject.tomlagainst the latest version on PyPI:- If
pyproject.toml > PyPI: builds the wheel + sdist and uploads viapoetry publish. - Otherwise: logs
::: skip — local X <= PyPI Yand exits cleanly.
- If
So a release = bump version in pyproject.toml, merge to main. Done.
Pipeline shape
verify ──► lint (ruff check + ruff format --check)
──► typecheck (pyright)
──► test (pytest)
publish ──► publish_pypi (only on main, only when version bumped)
All jobs run on the self-hosted EC2 GitLab runner (tags: [ec2-instance]).
PyPI authentication
poetry publish reads the POETRY_PYPI_TOKEN_PYPI env var (Poetry's standard
convention). It's configured as a GitLab CI/CD Variable on the project
(Settings → CI/CD → Variables): masked, scope = All environments.
Rotation = generate a new project-scoped token at https://pypi.org/manage/account/token/
(scope = reach-commons), paste into the same CI/CD Variable, revoke the old.
No restart or SSH needed.
Branches
| Branch | Purpose |
|---|---|
main |
source of truth; merges here trigger PyPI publish when version bumped |
fix/*, feat/*, chore/* |
working branches; verify pipeline runs but no publish |
Architecture cheat-sheet
consumer Lambda
│
├─ os.environ[lambda_function_api_key] ──┐
│ ▼
│ reach_commons.clients._auth.reach_api_bearer_header()
│ │
├─ EventProcessorClient ────────────────┤
├─ CallbackProcessorClient ────────────────┤ "Authorization": "Bearer <token>"
├─ ReachOpsApiClient ────────────────┤
└─ ReachDataBridgeClient ────────────────┘
Four client classes share the same Bearer token via a single helper — drop the
env var on the Lambda and all four start sending Bearer (empty) → 401s.
Test layout
All test files in tests/ use @pytest.mark.parametrize. Adding a non-parametrized
def test_x is discouraged — collapse into parameters or split if behaviors truly
diverge.
| File | Coverage |
|---|---|
test_imports.py |
parametrized over every reach_commons.* module via pkgutil.walk_packages — auto-discovers new submodules |
test_sms_smart_encoding.py |
GSM-7 / UCS-2 detection, normalization map, segment math at length boundaries (160/161/306/307, 70/71) |
test_validations.py |
Twilio HTTP wrapper status-code → bool fork; env var precedence (uppercase / lowercase / unset) |
test_auth.py |
reach_api_bearer_header() with token set / empty / unset; REACH_INTERNAL_API_KEY over SSM |
test_voice_contract.py |
voice contract version default, JSON round trips, int / string business_id |
test_outscraper.py |
env var → constructor → empty fallback resolution; X-API-KEY header |
Migration history
- 2026-05-12 — Migrated from AWS CodeCommit (
reach-commons→ renamed toreach-commons-MOVED-TO-GITLAB, read-only archive) to GitLab. Replaced manualpython build.pypublish with GitLab CI publish-on-version-bump. Removed hardcoded secrets from the wheel (Twilio creds, Bearer tokens shared across four internal-API clients, Outscraper token + webhook URL); all secrets now read from env vars sourced from/terraform/shared-config.tfSSM. Adopted ruff (replacing black + isort) and pyright; added parametrized test suite. Version 0.18.55 → 0.18.56.
Service ownership
Owner: Reach Engineering. Issues / MR reviews via the GitLab repo. The legacy
CodeCommit archive at reach-commons-MOVED-TO-GITLAB is read-only and kept
only for git archaeology.
4 maintainers
Metadata
4 maintainers
Release files for reach_commons 0.18.90
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| reach_commons-0.18.90.tar.gz | 52.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| reach_commons-0.18.90-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 115.1 kB
Release files / reach_commons-0.18.90.tar.gz
| Download URL | reach_commons-0.18.90.tar.gz |
|---|---|
| Size | 52.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9ef82f7294ce28c478509dccedccfba644d6c17d28ad51a90d1bfcfacd06df56
|
|
BLAKE2b-256 checksum How to use checksums |
62544dee316a4ace811cd8518e63fb9c80c2aab5c12f82e98a831959c20fedfe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.5 CPython/3.13.16 Linux/7.0.0-1012-aws
|
Release files / reach_commons-0.18.90-py3-none-any.whl
| Download URL | reach_commons-0.18.90-py3-none-any.whl |
|---|---|
| Size | 62.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
80c61a392699fa3c4b03bc7fc3c23a74d2403d340a6d70a84afbf270ad6fd8cc
|
|
BLAKE2b-256 checksum How to use checksums |
62772db4d07da7c20604ec3c27c47f563019e9e0e3d35b996280c912ffd76eee
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.5 CPython/3.13.16 Linux/7.0.0-1012-aws
|
