reach_commons 0.18.63

Reach Commons is a versatile utility library designed to streamline and enhance development workflows within the Reach ecosystem.

reach_commons

Reach's shared Python utility library — SMS encoding, phone validation, structured logging, persistence wrappers (Mongo, DynamoDB, S3, SQS, Firehose, KMS), Redis rate-limiting, and HTTP clients for internal Reach services (event-processor, callback-processor, reach-ops, reach-data-bridge), plus third-party integrations (HubSpot, Outscraper).

Distributed as the reach-commons wheel on PyPI. Source of truth: https://gitlab.com/reach.ai/reach-commons.

Install

pip install reach-commons

Pin a version in requirements.txt / pyproject.toml like any other dep. Releases are published from main by GitLab CI (see Release flow).

Quick example

from reach_commons.sms_smart_encoding import MessageSmartEncoding

msg = MessageSmartEncoding("Hello world!")
print(msg.encoding, msg.length, msg.segments)  # GSM-7 12 1

Repo layout

reach_commons/
├── app_logging/          structured logger, http logger, deprecation tracker
├── clients/              thin HTTP wrappers for Reach internal services + 3rd-party APIs
│   ├── _auth.py            shared Bearer-header helper (reads lambda_function_api_key)
│   ├── callback_processor.py
│   ├── event_processor.py
│   ├── hubspot.py
│   ├── nightly_lambda.py
│   ├── outscraper.py
│   ├── reach_data_bridge.py
│   └── reach_ops_api.py
├── mongo/                MongoDB customer persistence (sync + async)
├── reach_aws/            boto3 wrappers (dynamodb, s3, sqs, firehose, kms, rate limiter)
├── reach_base_model.py
├── redis_manager.py
├── sms_smart_encoding.py   GSM-7 / UCS-2 detection + segment math
├── utils.py
└── validations.py          phone-number validator (Twilio Lookups)
tests/                   pytest, every suite uses @pytest.mark.parametrize

Security model

Starting with version 0.18.57, the wheel published to PyPI contains zero credentials. Every secret consumed by reach_commons is fetched at runtime from AWS Systems Manager Parameter Store (SecureString), sourced from /terraform/shared-config.tf (shared_ssm_secure).

Why this matters: reach_commons is published to public PyPI. Anyone can pip download reach-commons. The wheel is intended to be safe in that posture — if it gets shared outside Reach (forked, cloned by an external dev, included in a third-party tool), no Reach credentials travel with it.

Access control lives in IAM: a consumer Lambda can only read the secrets its execution role grants ssm:GetParameter on. No env vars, no hardcoded fallbacks, no setattr side-channels.

SSM paths consumed at runtime

SSM path	Read by	Source line
/api-keys/{env}/lambda-function-api-key	_auth.reach_api_bearer_header() — Bearer for EventProcessorClient, CallbackProcessorClient, ReachOpsApiClient, ReachDataBridgeClient	/terraform/shared-config.tf:19
/twilio/{env}/account_sid	CommonsPhoneNumberValidation	/terraform/shared-config.tf:22
/twilio/{env}/auth_token	CommonsPhoneNumberValidation	/terraform/shared-config.tf:23
/outscraper/{env}/token	OutscraperClient	/terraform/shared-config.tf:24
/outscraper/{env}/webhook_url	OutscraperClient	/terraform/shared-config.tf:25

{env} resolves to staging / prod from the ENV Lambda env var (defaults to Staging).

Required consumer Lambda IAM

Every consumer must grant its execution role ssm:GetParameter on the paths it touches (only the ones whose clients it actually imports). Example for a Lambda that uses Bearer + Twilio + Outscraper:

statement {
  sid     = "ReachCommonsSSMRead"
  effect  = "Allow"
  actions = ["ssm:GetParameter", "ssm:GetParameters"]
  resources = [
    "arn:aws:ssm:us-east-1:383836045505:parameter/api-keys/${lower(var.tag)}/*",
    "arn:aws:ssm:us-east-1:383836045505:parameter/twilio/${lower(var.tag)}/*",
    "arn:aws:ssm:us-east-1:383836045505:parameter/outscraper/${lower(var.tag)}/*",
  ]
}

If the SSM SecureStrings are encrypted with a non-default KMS CMK, add kms:Decrypt on that key alias too. The current shared_ssm_secure block uses the default alias/aws/ssm so no extra KMS statement is needed.

Caching and rotation

reach_commons._ssm caches each SSM value in-memory with a 1-hour TTL per process. Cold start = 1 GetParameter per credential (~50ms each); warm container reuses the cached value until expiry.

After 1h, a rotation in SSM is picked up automatically on the next call — no redeploy needed. To force an immediate refresh, redeploy / recycle the Lambda containers.

Test / local override

Each affected client accepts explicit kwargs so tests and local tooling can bypass SSM entirely:

CommonsPhoneNumberValidation(account_sid="ACtest", auth_token="...")
OutscraperClient(token="...", webhook_url="https://...")

reach_api_bearer_header() has no kwarg path — tests should monkeypatch.setattr(reach_commons.clients._auth, "get_secure", lambda p: "...").

Local development

poetry install                       # runtime + dev deps into .venv
poetry run pre-commit install        # one-time: enables commit/push hooks

poetry run pytest                    # full suite
poetry run ruff check reach_commons tests
poetry run ruff format --check reach_commons tests
poetry run pyright reach_commons tests

Python floor: ^3.8. CI runs on python:3.13-slim.

Pre-commit hooks

.pre-commit-config.yaml installs three layers:

• On commit (fast): trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-merge-conflict, check-added-large-files, ruff (lint + autofix), ruff-format.
• On push (slower): pyright with the runtime deps as additional_dependencies.

Pyright runs only on push so day-to-day commits stay fast.

Release flow

GitLab CI publishes to PyPI automatically. Workflow:

1. Open an MR against main.
2. CI runs lint + typecheck + test on every push (verify stage).
3. Merge to main.
4. The publish_pypi job compares the version in pyproject.toml against the latest version on PyPI:
   • If pyproject.toml > PyPI: builds the wheel + sdist and uploads via poetry publish.
   • Otherwise: logs ::: skip — local X <= PyPI Y and exits cleanly.

So a release = bump version in pyproject.toml, merge to main. Done.

Pipeline shape

verify ──► lint        (ruff check + ruff format --check)
       ──► typecheck   (pyright)
       ──► test        (pytest)
publish ──► publish_pypi   (only on main, only when version bumped)

All jobs run on the self-hosted EC2 GitLab runner (tags: [ec2-instance]).

PyPI authentication

poetry publish reads the POETRY_PYPI_TOKEN_PYPI env var (Poetry's standard convention). It's configured as a GitLab CI/CD Variable on the project (Settings → CI/CD → Variables): masked, scope = All environments.

Rotation = generate a new project-scoped token at https://pypi.org/manage/account/token/ (scope = reach-commons), paste into the same CI/CD Variable, revoke the old. No restart or SSH needed.

Branches

Branch	Purpose
main	source of truth; merges here trigger PyPI publish when version bumped
fix/*, feat/*, chore/*	working branches; verify pipeline runs but no publish

Architecture cheat-sheet

consumer Lambda
   │
   ├─ os.environ[lambda_function_api_key]  ──┐
   │                                          ▼
   │                     reach_commons.clients._auth.reach_api_bearer_header()
   │                                          │
   ├─ EventProcessorClient    ────────────────┤
   ├─ CallbackProcessorClient ────────────────┤   "Authorization": "Bearer <token>"
   ├─ ReachOpsApiClient       ────────────────┤
   └─ ReachDataBridgeClient   ────────────────┘

Four client classes share the same Bearer token via a single helper — drop the env var on the Lambda and all four start sending Bearer (empty) → 401s.

Test layout

All test files in tests/ use @pytest.mark.parametrize. Adding a non-parametrized def test_x is discouraged — collapse into parameters or split if behaviors truly diverge.

File	Coverage
test_imports.py	parametrized over every reach_commons.* module via pkgutil.walk_packages — auto-discovers new submodules
test_sms_smart_encoding.py	GSM-7 / UCS-2 detection, normalization map, segment math at length boundaries (160/161/306/307, 70/71)
test_validations.py	Twilio HTTP wrapper status-code → bool fork; env var precedence (uppercase / lowercase / unset)
test_auth.py	reach_api_bearer_header() with token set / empty / unset
test_outscraper.py	env var → constructor → empty fallback resolution; X-API-KEY header

Migration history

• 2026-05-12 — Migrated from AWS CodeCommit (reach-commons → renamed to reach-commons-MOVED-TO-GITLAB, read-only archive) to GitLab. Replaced manual python build.py publish with GitLab CI publish-on-version-bump. Removed hardcoded secrets from the wheel (Twilio creds, Bearer tokens shared across four internal-API clients, Outscraper token + webhook URL); all secrets now read from env vars sourced from /terraform/shared-config.tf SSM. Adopted ruff (replacing black + isort) and pyright; added parametrized test suite. Version 0.18.55 → 0.18.56.

Service ownership

Owner: Reach Engineering. Issues / MR reviews via the GitLab repo. The legacy CodeCommit archive at reach-commons-MOVED-TO-GITLAB is read-only and kept only for git archaeology.