reach_commons 0.18.56

Reach Commons is a versatile utility library designed to streamline and enhance development workflows within the Reach ecosystem.

reach_commons

Reach's shared Python utility library — SMS encoding, phone validation, structured logging, persistence wrappers (Mongo, DynamoDB, S3, SQS, Firehose, KMS), Redis rate-limiting, and HTTP clients for internal Reach services (event-processor, callback-processor, reach-ops, reach-data-bridge), plus third-party integrations (HubSpot, Outscraper).

Distributed as the reach-commons wheel on PyPI. Source of truth: https://gitlab.com/reach.ai/reach-commons.

Install

pip install reach-commons

Pin a version in requirements.txt / pyproject.toml like any other dep. Releases are published from main by GitLab CI (see Release flow).

Quick example

from reach_commons.sms_smart_encoding import MessageSmartEncoding

msg = MessageSmartEncoding("Hello world!")
print(msg.encoding, msg.length, msg.segments)  # GSM-7 12 1

Repo layout

reach_commons/
├── app_logging/          structured logger, http logger, deprecation tracker
├── clients/              thin HTTP wrappers for Reach internal services + 3rd-party APIs
│   ├── _auth.py            shared Bearer-header helper (reads lambda_function_api_key)
│   ├── callback_processor.py
│   ├── event_processor.py
│   ├── hubspot.py
│   ├── nightly_lambda.py
│   ├── outscraper.py
│   ├── reach_data_bridge.py
│   └── reach_ops_api.py
├── mongo/                MongoDB customer persistence (sync + async)
├── reach_aws/            boto3 wrappers (dynamodb, s3, sqs, firehose, kms, rate limiter)
├── reach_base_model.py
├── redis_manager.py
├── sms_smart_encoding.py   GSM-7 / UCS-2 detection + segment math
├── utils.py
└── validations.py          phone-number validator (Twilio Lookups)
tests/                   pytest, every suite uses @pytest.mark.parametrize

Required environment variables for consumers

Starting with version 0.18.56, secrets are no longer baked into the wheel. Every credential is read at runtime from environment variables. Consumer lambdas must inject these before instantiating any of the affected client classes; unset values fall back to "" and produce loud 401s from upstream services rather than silently working with stale defaults.

Env var	Read by	Source of truth
lambda_function_api_key	_auth.reach_api_bearer_header() — used by EventProcessorClient, CallbackProcessorClient, ReachOpsApiClient, ReachDataBridgeClient	SSM SecureString /api-keys/{env}/lambda-function-api-key (/terraform/shared-config.tf:19)
TWILIO_ACCOUNT_SID or twilio_account_sid	CommonsPhoneNumberValidation	SSM /twilio/{env}/account_sid (/terraform/shared-config.tf:22)
TWILIO_AUTH_TOKEN or twilio_auth_token	CommonsPhoneNumberValidation	SSM /twilio/{env}/auth_token (/terraform/shared-config.tf:23)
outscraper_token	OutscraperClient	SSM /outscraper/{env}/token (/terraform/shared-config.tf:24)
outscraper_webhook_url	OutscraperClient	SSM /outscraper/{env}/webhook_url (/terraform/shared-config.tf:25)
ENV	several clients to pick base_url ("Staging" / "Prod")	usually set by Terraform Lambda config
HUBSPOT_BASE_URL (optional)	HubSpotClient constructor default	env-injected

Twilio is read with uppercase preferred, lowercase fallback — covers reach-data-bridge-lambda / reach-ops-api-lambda (uppercase) and sms-processor-lambda (lowercase), no consumer churn required.

Wiring: each consumer Lambda's serverless.yml must declare the env var and map it to the SSM parameter published by /terraform/shared-config.tf (or to its own per-lambda SSM resource). See POS-API-Puller-lambda/infra/env.tf for the modern puller-style pattern; legacy lambdas use Serverless Framework variable resolvers.

Local development

poetry install                       # runtime + dev deps into .venv
poetry run pre-commit install        # one-time: enables commit/push hooks

poetry run pytest                    # full suite
poetry run ruff check reach_commons tests
poetry run ruff format --check reach_commons tests
poetry run pyright reach_commons tests

Python floor: ^3.8. CI runs on python:3.13-slim.

Pre-commit hooks

.pre-commit-config.yaml installs three layers:

• On commit (fast): trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-merge-conflict, check-added-large-files, ruff (lint + autofix), ruff-format.
• On push (slower): pyright with the runtime deps as additional_dependencies.

Pyright runs only on push so day-to-day commits stay fast.

Release flow

GitLab CI publishes to PyPI automatically. Workflow:

1. Open an MR against main.
2. CI runs lint + typecheck + test on every push (verify stage).
3. Merge to main.
4. The publish_pypi job compares the version in pyproject.toml against the latest version on PyPI:
   • If pyproject.toml > PyPI: builds the wheel + sdist and uploads via poetry publish.
   • Otherwise: logs ::: skip — local X <= PyPI Y and exits cleanly.

So a release = bump version in pyproject.toml, merge to main. Done.

Pipeline shape

verify ──► lint        (ruff check + ruff format --check)
       ──► typecheck   (pyright)
       ──► test        (pytest)
publish ──► publish_pypi   (only on main, only when version bumped)

All jobs run on the self-hosted EC2 GitLab runner (tags: [ec2-instance]).

PyPI authentication

poetry publish reads the POETRY_PYPI_TOKEN_PYPI env var (Poetry's standard convention). The token is injected at the gitlab-runner level, not as a GitLab CI/CD variable — see /etc/gitlab-runner/config.toml on the runner host:

[[runners]]
  name = "reach-ec2-runner"
  environment = [
    "POETRY_PYPI_TOKEN_PYPI=pypi-AgEIcHlwa…"
  ]

This keeps the token out of GitLab project settings entirely; rotation = SSH to the runner, edit the line, systemctl restart gitlab-runner.

Branches

Branch	Purpose
main	source of truth; merges here trigger PyPI publish when version bumped
fix/*, feat/*, chore/*	working branches; verify pipeline runs but no publish

Architecture cheat-sheet

consumer Lambda
   │
   ├─ os.environ[lambda_function_api_key]  ──┐
   │                                          ▼
   │                     reach_commons.clients._auth.reach_api_bearer_header()
   │                                          │
   ├─ EventProcessorClient    ────────────────┤
   ├─ CallbackProcessorClient ────────────────┤   "Authorization": "Bearer <token>"
   ├─ ReachOpsApiClient       ────────────────┤
   └─ ReachDataBridgeClient   ────────────────┘

Four client classes share the same Bearer token via a single helper — drop the env var on the Lambda and all four start sending Bearer (empty) → 401s.

Test layout

All test files in tests/ use @pytest.mark.parametrize. Adding a non-parametrized def test_x is discouraged — collapse into parameters or split if behaviors truly diverge.

File	Coverage
test_imports.py	parametrized over every reach_commons.* module via pkgutil.walk_packages — auto-discovers new submodules
test_sms_smart_encoding.py	GSM-7 / UCS-2 detection, normalization map, segment math at length boundaries (160/161/306/307, 70/71)
test_validations.py	Twilio HTTP wrapper status-code → bool fork; env var precedence (uppercase / lowercase / unset)
test_auth.py	reach_api_bearer_header() with token set / empty / unset
test_outscraper.py	env var → constructor → empty fallback resolution; X-API-KEY header

Migration history

• 2026-05-12 — Migrated from AWS CodeCommit (reach-commons → renamed to reach-commons-MOVED-TO-GITLAB, read-only archive) to GitLab. Replaced manual python build.py publish with GitLab CI publish-on-version-bump. Removed hardcoded secrets from the wheel (Twilio creds, Bearer tokens shared across four internal-API clients, Outscraper token + webhook URL); all secrets now read from env vars sourced from /terraform/shared-config.tf SSM. Adopted ruff (replacing black + isort) and pyright; added parametrized test suite. Version 0.18.55 → 0.18.56.

Service ownership

Owner: Reach Engineering. Issues / MR reviews via the GitLab repo. The legacy CodeCommit archive at reach-commons-MOVED-TO-GITLAB is read-only and kept only for git archaeology.