Metadata-Version: 2.1
Name: reach_commons
Version: 0.18.79
Summary: Reach Commons is a versatile utility library designed to streamline and enhance development workflows within the Reach ecosystem.
License: MIT
Author: Engineering
Author-email: engineering@getreach.ai
Requires-Python: >=3.8,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: boto3 (>=1.28.68)
Requires-Dist: curlify (==3.0.0)
Requires-Dist: fastapi (>=0.115.5)
Requires-Dist: pydantic (>=2.9.2)
Description-Content-Type: text/markdown

# reach_commons

Reach's shared Python utility library — SMS encoding, phone validation, structured
logging, persistence wrappers (Mongo, DynamoDB, S3, SQS, Firehose, KMS), Redis
rate-limiting, and HTTP clients for internal Reach services (event-processor,
callback-processor, reach-ops, reach-data-bridge), plus third-party integrations
(HubSpot, Outscraper).

Distributed as the [`reach-commons`](https://pypi.org/project/reach-commons/) wheel on
PyPI. Source of truth: <https://gitlab.com/reach.ai/reach-commons>.

## Install

```bash
pip install reach-commons
```

Pin a version in `requirements.txt` / `pyproject.toml` like any other dep. Releases
are published from `main` by GitLab CI (see [Release flow](#release-flow)).

## Quick example

```python
from reach_commons.sms_smart_encoding import MessageSmartEncoding

msg = MessageSmartEncoding("Hello world!")
print(msg.encoding, msg.length, msg.segments)  # GSM-7 12 1
```

## Repo layout

```
reach_commons/
├── app_logging/          structured logger, http logger, deprecation tracker
├── clients/              thin HTTP wrappers for Reach internal services + 3rd-party APIs
│   ├── _auth.py            shared Bearer-header helper (reads lambda_function_api_key)
│   ├── callback_processor.py
│   ├── event_processor.py
│   ├── hubspot.py
│   ├── nightly_lambda.py
│   ├── outscraper.py
│   ├── reach_data_bridge.py
│   └── reach_ops_api.py
├── credentials.py        one named accessor per Parameter Store credential
├── mongo/                MongoDB customer persistence (sync + async)
├── reach_aws/            boto3 wrappers (dynamodb, s3, sqs, firehose, kms, rate limiter)
├── reach_base_model.py
├── redis_manager.py
├── sms_smart_encoding.py   GSM-7 / UCS-2 detection + segment math
├── utils.py
└── validations.py          phone-number validator (Twilio Lookups)
tests/                   pytest, every suite uses @pytest.mark.parametrize
```

## Security model

Starting with version **0.18.57**, **the wheel published to PyPI contains zero
credentials**. Every secret consumed by reach_commons is fetched at runtime from
AWS Systems Manager Parameter Store (SecureString), sourced from
`/terraform/shared-config.tf` (`shared_ssm_secure`).

Why this matters: reach_commons is published to **public** PyPI. Anyone can
`pip download reach-commons`. The wheel is intended to be safe in that posture
— if it gets shared outside Reach (forked, cloned by an external dev, included
in a third-party tool), no Reach credentials travel with it.

Access control lives in **IAM**: a consumer Lambda can only read the secrets
its execution role grants `ssm:GetParameter` on. No env vars, no hardcoded
fallbacks, no `setattr` side-channels.

### Credential accessors

Every credential kept in Parameter Store has one named accessor in
`reach_commons/credentials.py`, and that module is the only place its path is written.
A value shared by several services is one parameter named after its vendor
(`/stripe/{env}/secret-key`), never after a consumer, so rotating it is one write.

```python
from reach_commons import credentials

credentials.preload(credentials.stripe_secret_key, credentials.sendgrid_api_key)


def handler(event, context):
    stripe.api_key = credentials.stripe_secret_key()
```

`preload(...)` at module import resolves the credentials during the cold start in one
batched `GetParameters` call (ten names per call), so a missing IAM grant or parameter
raises there instead of on the first request, and a deploy that starts many containers
at once does not exhaust Parameter Store's request rate. Handlers
then call the accessor itself, which reads the in-process cache and picks up a rotation
within the TTL. Do not copy a value into a module-level constant: that pins it for the
life of the container.

`{env}` resolves to `staging` / `prod` from the `ENV` Lambda env var (defaults to
`Staging`). The commons clients read theirs the same way: `_auth` uses
`reach_internal_api_key`, `CommonsPhoneNumberValidation` the Twilio pair, and
`OutscraperClient` the Outscraper token and webhook URL.

### Required consumer Lambda IAM

Grant `ssm:GetParameter` on each exact parameter the consumer reads, in its own
environment only. Never a prefix and never `GetParametersByPath`: a by-path read is how
one service once pulled the other environment's secrets into its state.

```hcl
statement {
  sid     = "ReadOwnCredentials"
  effect  = "Allow"
  actions = ["ssm:GetParameter", "ssm:GetParameters"]
  resources = [
    "arn:aws:ssm:us-east-1:383836045505:parameter/api-keys/${lower(var.tag)}/lambda-function-api-key",
    "arn:aws:ssm:us-east-1:383836045505:parameter/stripe/${lower(var.tag)}/secret-key",
  ]
}
```

The parameters use the default `alias/aws/ssm` key. `/terraform/ssm-credential-reads.tf`
is the reference for the lambda roles in that repo.

### Caching and rotation

`reach_commons._ssm` caches each SSM value in-memory with a **1-hour TTL** per
process. Cold start = 1 GetParameter per credential (~50ms each); warm
container reuses the cached value until expiry.

After 1h, a rotation in SSM is picked up automatically on the next call — no
redeploy needed. To force an immediate refresh, redeploy / recycle the Lambda
containers.

### Test / local override

Test suites replace Parameter Store before importing code that preloads credentials:

```python
import reach_commons.credentials

reach_commons.credentials.override_for_tests(lambda path: f"test-value-for:{path}")
```


Each affected client accepts explicit kwargs so tests and local tooling can
bypass SSM entirely:

```python
CommonsPhoneNumberValidation(account_sid="ACtest", auth_token="...")
OutscraperClient(token="...", webhook_url="https://...")
```

`reach_api_bearer_header()` has no kwarg path — tests should
`monkeypatch.setattr(reach_commons.clients._auth, "get_secure", lambda p: "...")`.

## Local development

```bash
poetry install                       # runtime + dev deps into .venv
poetry run pre-commit install        # one-time: enables commit/push hooks

poetry run pytest                    # full suite
poetry run ruff check reach_commons tests
poetry run ruff format --check reach_commons tests
poetry run pyright reach_commons tests
```

Python floor: `^3.8`. CI runs on `python:3.13-slim`.

### Pre-commit hooks

`.pre-commit-config.yaml` installs three layers:

- **On commit** (fast): trailing-whitespace, end-of-file-fixer, check-yaml, check-toml,
  check-merge-conflict, check-added-large-files, **ruff** (lint + autofix), **ruff-format**.
- **On push** (slower): **pyright** with the runtime deps as `additional_dependencies`.

Pyright runs only on push so day-to-day commits stay fast.

## Release flow

GitLab CI publishes to PyPI automatically. Workflow:

1. Open an MR against `main`.
2. CI runs `lint` + `typecheck` + `test` on every push (verify stage).
3. Merge to `main`.
4. The `publish_pypi` job compares the `version` in `pyproject.toml` against
   the latest version on PyPI:
   - If `pyproject.toml > PyPI`: builds the wheel + sdist and uploads via
     `poetry publish`.
   - Otherwise: logs `::: skip — local X <= PyPI Y` and exits cleanly.

So a release = **bump `version` in `pyproject.toml`, merge to `main`**. Done.

### Pipeline shape

```
verify ──► lint        (ruff check + ruff format --check)
       ──► typecheck   (pyright)
       ──► test        (pytest)
publish ──► publish_pypi   (only on main, only when version bumped)
```

All jobs run on the self-hosted EC2 GitLab runner (`tags: [ec2-instance]`).

### PyPI authentication

`poetry publish` reads the `POETRY_PYPI_TOKEN_PYPI` env var (Poetry's standard
convention). It's configured as a **GitLab CI/CD Variable** on the project
(`Settings → CI/CD → Variables`): masked, scope = `All environments`.

Rotation = generate a new project-scoped token at <https://pypi.org/manage/account/token/>
(scope = `reach-commons`), paste into the same CI/CD Variable, revoke the old.
No restart or SSH needed.

### Branches

| Branch | Purpose |
|---|---|
| `main` | source of truth; merges here trigger PyPI publish when version bumped |
| `fix/*`, `feat/*`, `chore/*` | working branches; verify pipeline runs but no publish |

## Architecture cheat-sheet

```
consumer Lambda
   │
   ├─ os.environ[lambda_function_api_key]  ──┐
   │                                          ▼
   │                     reach_commons.clients._auth.reach_api_bearer_header()
   │                                          │
   ├─ EventProcessorClient    ────────────────┤
   ├─ CallbackProcessorClient ────────────────┤   "Authorization": "Bearer <token>"
   ├─ ReachOpsApiClient       ────────────────┤
   └─ ReachDataBridgeClient   ────────────────┘
```

Four client classes share the same Bearer token via a single helper — drop the
env var on the Lambda and all four start sending `Bearer ` (empty) → 401s.

## Test layout

All test files in `tests/` use `@pytest.mark.parametrize`. Adding a non-parametrized
`def test_x` is discouraged — collapse into parameters or split if behaviors truly
diverge.

| File | Coverage |
|---|---|
| `test_imports.py` | parametrized over every `reach_commons.*` module via `pkgutil.walk_packages` — auto-discovers new submodules |
| `test_sms_smart_encoding.py` | GSM-7 / UCS-2 detection, normalization map, segment math at length boundaries (160/161/306/307, 70/71) |
| `test_validations.py` | Twilio HTTP wrapper status-code → bool fork; env var precedence (uppercase / lowercase / unset) |
| `test_auth.py` | `reach_api_bearer_header()` with token set / empty / unset |
| `test_outscraper.py` | env var → constructor → empty fallback resolution; `X-API-KEY` header |

## Migration history

- **2026-05-12** — Migrated from AWS CodeCommit (`reach-commons` → renamed to
  `reach-commons-MOVED-TO-GITLAB`, read-only archive) to GitLab. Replaced manual
  `python build.py` publish with GitLab CI publish-on-version-bump. Removed
  hardcoded secrets from the wheel (Twilio creds, Bearer tokens shared across
  four internal-API clients, Outscraper token + webhook URL); all secrets now
  read from env vars sourced from `/terraform/shared-config.tf` SSM. Adopted
  ruff (replacing black + isort) and pyright; added parametrized test suite.
  Version 0.18.55 → 0.18.56.

## Service ownership

Owner: Reach Engineering. Issues / MR reviews via the GitLab repo. The legacy
CodeCommit archive at `reach-commons-MOVED-TO-GITLAB` is read-only and kept
only for git archaeology.

