Metadata-Version: 2.4
Name: dtpyfw
Version: 1.14
Summary: DealerTower Python Framework: reusable building‑blocks for DealerTower services
License: DealerTower Proprietary License
License-File: LICENSE
Keywords: framework,microservices,fastapi,sqlalchemy,celery,redis,kafka,opensearch
Author: Reza Shirazi
Author-email: reza@dealertower.com
Requires-Python: >=3.13,<4.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Framework :: Celery
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Provides-Extra: all
Provides-Extra: api
Provides-Extra: bucket
Provides-Extra: db
Provides-Extra: db-mysql
Provides-Extra: encrypt
Provides-Extra: ftp
Provides-Extra: kafka
Provides-Extra: mcp
Provides-Extra: normal
Provides-Extra: opensearch
Provides-Extra: redis
Provides-Extra: redis-streamer
Provides-Extra: slim-api
Provides-Extra: slim-task
Provides-Extra: worker
Requires-Dist: PyMySQL (>=1.1.2,<1.2.0) ; extra == "db-mysql"
Requires-Dist: SQLAlchemy[asyncio] (>=2.0.49,<2.1.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "slim-task" or extra == "db" or extra == "db-mysql"
Requires-Dist: aiohttp[speedups] (>=3.13.5,<3.14.0)
Requires-Dist: aiomysql (>=0.3.2,<0.4.0) ; extra == "db-mysql"
Requires-Dist: asyncpg (>=0.31.0,<0.32.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "slim-task" or extra == "db"
Requires-Dist: bcrypt (>=5.0.0,<5.1.0) ; extra == "all" or extra == "encrypt"
Requires-Dist: boto3 (>=1.42.0,<1.43.0) ; extra == "all" or extra == "bucket"
Requires-Dist: celery-redbeat (>=2.3.3,<2.4.0) ; extra == "all" or extra == "normal" or extra == "slim-task" or extra == "worker"
Requires-Dist: celery[brotli] (>=5.6.3,<5.7.0) ; extra == "all" or extra == "normal" or extra == "slim-task" or extra == "worker"
Requires-Dist: celery_once (>=3.0.1,<3.1.0) ; extra == "all" or extra == "normal" or extra == "slim-task" or extra == "worker"
Requires-Dist: fastapi (>=0.136.1,<0.137.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "api"
Requires-Dist: fastmcp (>=2.14.0,<2.15.0) ; extra == "all" or extra == "mcp"
Requires-Dist: gunicorn (>=25.3.0,<25.4.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "api"
Requires-Dist: httpx (>=0.28.1,<0.29.0) ; extra == "all" or extra == "mcp"
Requires-Dist: kafka-python (>=2.3.1,<2.4.0) ; extra == "all" or extra == "kafka"
Requires-Dist: opensearch-py[async] (>=3.1.0,<3.2.0) ; extra == "all" or extra == "opensearch"
Requires-Dist: paramiko (>=4.0.0,<4.1.0) ; extra == "all" or extra == "ftp"
Requires-Dist: passlib[argon2,bcrypt] (>=1.7.4,<1.8.0) ; extra == "all" or extra == "encrypt"
Requires-Dist: psycopg2 (>=2.9.12,<2.10.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "slim-task" or extra == "db"
Requires-Dist: pydantic (>=2.13.3,<2.14.0)
Requires-Dist: python-dateutil (>=2.9.0,<2.10.0) ; extra == "all" or extra == "ftp"
Requires-Dist: python-jose (>=3.5.0,<3.6.0) ; extra == "all" or extra == "encrypt"
Requires-Dist: python-multipart (>=0.0.26,<0.1.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "api"
Requires-Dist: redis[hiredis] (>=7.4.0,<7.5.0) ; extra == "all" or extra == "normal" or extra == "slim-task" or extra == "redis" or extra == "redis-streamer" or extra == "worker"
Requires-Dist: requests (>=2.33.1,<2.34.0)
Requires-Dist: uvicorn[standard] (>=0.46.0,<0.47.0) ; extra == "all" or extra == "normal" or extra == "slim-api" or extra == "api"
Project-URL: Documentation, https://github.com/datgate/dtpyfw/tree/main/docs
Project-URL: Homepage, https://github.com/datgate/dtpyfw
Project-URL: Repository, https://github.com/datgate/dtpyfw
Description-Content-Type: text/markdown

# DealerTower Python Framework (dtpyfw)

[![Python Version](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://www.python.org/downloads/)
[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Type Checked](https://img.shields.io/badge/type%20checked-mypy-blue.svg)](http://mypy-lang.org/)
[![License](https://img.shields.io/badge/license-Proprietary-red.svg)](LICENSE)

**DealerTower Python Framework (dtpyfw)** is a production-ready internal framework providing reusable building blocks for DealerTower microservices. It covers API development, database orchestration, caching, event streaming, object storage, task scheduling, and structured logging — all with full type safety and consistent interfaces.

PEP 561 typed package — `py.typed` marker is shipped so mypy/pyright narrow types in consumer services automatically.

---

## Installation

Requires **Python 3.13** or newer.

### Base

```bash
pip install dtpyfw
```

Includes `dtpyfw.core` (env, retry, hashing, chunking, validation, …) and `dtpyfw.log` (structured logging).

### From git tag (recommended for internal services)

```text
dtpyfw @ git+https://github.com/datgate/dtpyfw.git@v1.0
```

### Development

```bash
poetry install -E all
```

### Check installed version

```python
import dtpyfw
print(dtpyfw.__version__)  # "1.0"
```

### Optional extras

Extras can be combined: `pip install dtpyfw[api,db,redis]`.

| Extra | Key dependencies | Install |
|---|---|---|
| `api` | FastAPI, Uvicorn, Gunicorn | `pip install dtpyfw[api]` |
| `db` | SQLAlchemy 2, asyncpg, psycopg2 | `pip install dtpyfw[db]` |
| `db-mysql` | PyMySQL, aiomysql | `pip install dtpyfw[db-mysql]` |
| `bucket` | boto3 | `pip install dtpyfw[bucket]` |
| `redis` | redis-py + hiredis | `pip install dtpyfw[redis]` |
| `redis_streamer` | redis-py | `pip install dtpyfw[redis_streamer]` |
| `worker` | Celery, celery-redbeat, celery_once | `pip install dtpyfw[worker]` |
| `kafka` | kafka-python | `pip install dtpyfw[kafka]` |
| `opensearch` | opensearch-py | `pip install dtpyfw[opensearch]` |
| `ftp` | paramiko | `pip install dtpyfw[ftp]` |
| `encrypt` | python-jose, passlib, bcrypt | `pip install dtpyfw[encrypt]` |
| `all` | Everything above | `pip install dtpyfw[all]` |

Common profiles:

```bash
pip install dtpyfw[api,db,redis]        # API microservice
pip install dtpyfw[worker,db,redis]     # Celery worker service
pip install dtpyfw[api,db,opensearch,redis]  # Search-enabled service
pip install dtpyfw[bucket,db,ftp]       # Data processing service
```

---

## Documentation

- **[CHANGELOG.md](CHANGELOG.md)** — version index linking to per-release notes in [`docs/changelogs/`](docs/changelogs/)
- **[UPGRADE.md](UPGRADE.md)** — migration guides linking to [`docs/upgrades/`](docs/upgrades/)
- **[docs/](docs/)** — detailed module documentation for every subpackage

---

## Quick start examples

### FastAPI application

```python
from dtpyfw.api import Application
from dtpyfw.api.routes import Router, Route, RouteMethod
from dtpyfw.api.routes.authentication import Auth, AuthType
from dtpyfw.core.env import Env

gateway_auth = Auth.from_env(
    auth_type=AuthType.HEADER,
    header_key="x-gateway-key",
    env_var="gateway_key",
)

router = Router(prefix="/health", tags=["health"])
router.add_route(Route(
    method=RouteMethod.GET,
    path="/",
    endpoint=lambda: {"status": "ok"},
))

app = Application(
    title="My Microservice",
    version="1.0",
    routers=[router],
    auth=gateway_auth,
).get_app()
```

### Database

```python
from dtpyfw.db.config import DatabaseConfig
from dtpyfw.db.database import DatabaseInstance

db_config = DatabaseConfig.from_env()   # reads db_host, db_port, db_user, …
db = DatabaseInstance(db_config)

with db.get_db_cm_sync() as session:
    result = session.execute(select(User)).scalars().all()
```

### Structured logging

```python
from dtpyfw.log.config import LogConfig
from dtpyfw.log.initializer import log_initializer

log_config = LogConfig.from_env()   # reads logging_ms_url, log_level, …
log_initializer(config=log_config)

# Inside any function:
from dtpyfw.log import footprint

footprint.leave(
    log_type="info",
    controller=f"{__name__}.my_func",
    subject="Task started",
    message="Processing item.",
    payload={"item_id": item_id},
)
```

### Redis caching

```python
from dtpyfw.redis.config import RedisConfig
from dtpyfw.redis.connection import RedisInstance

redis = RedisInstance(RedisConfig.from_env())   # reads redis_url / redis_host, …

from dtpyfw.redis.caching import cache_function

@cache_function(redis_instance=redis, expire_time=3600)
def get_dealer(dealer_id: str) -> dict:
    ...
```

### S3-compatible storage

```python
from dtpyfw.bucket.bucket import Bucket

bucket = Bucket.from_env()          # reads s3_bucket_name, s3_access_key, …
bucket = Bucket.from_env("media_s3_")  # custom prefix

url = bucket.upload("path/to/file.pdf", "dealers/123/file.pdf")
bucket.download("dealers/123/file.pdf", "/tmp/file.pdf")
exists = bucket.exists("dealers/123/file.pdf")
```

### Celery worker

```python
from dtpyfw.worker.task import Task
from dtpyfw.worker.worker import Worker
from dtpyfw.redis.config import RedisConfig
from dtpyfw.redis.connection import RedisInstance

task = Task()
task.register("myapp.tasks.process_data", queue="default")
task.add_periodic("myapp.tasks.cleanup", crontab(hour=0, minute=0))

redis = RedisInstance(RedisConfig.from_env())
worker = Worker()
worker.set_name("my_worker").set_redis(redis).set_task(task)
celery_app = worker.get_celery()
```

### Startup diagnostics report

Emit a structured boot-time snapshot (environment variables, dependency
health, process metadata) to stdout and `footprint`.  All dependency
probes run concurrently; the function never raises regardless of which
probes fail.

```python
from dtpyfw.diagnostics import emit_startup_report

# Minimal (process info + env snapshot, no dependency probes)
emit_startup_report(service_name="my-service", process_role="api")

# With dependencies
from app.config.database import database
from app.config.redis import redis_instance, redis_queue_instance
from app.config.log import log_config

emit_startup_report(
    service_name="my-service",
    process_role="api",
    service_version="1.4.2",
    database=database,
    redis=[redis_instance, redis_queue_instance],
    log_config=log_config,
    probe_timeout_seconds=3.0,
)

# Collect without emitting
from dtpyfw.diagnostics import collect_startup_report
import json

report = collect_startup_report(service_name="my-service", process_role="script")
print(json.dumps(report, indent=2, default=str))
```

The returned dict always has `schema_version=1` and the following
top-level keys: `status` (`"ok"` / `"degraded"` / `"down"`), `process`,
`environment`, `logging`, `dependencies`, `extras`, `notes`.

---

### Redis Streams (event bus)

```python
from dtpyfw.redis.config import RedisConfig
from dtpyfw.redis.connection import RedisInstance
from dtpyfw.redis_streamer.synchronize import RedisStreamer
from dtpyfw.redis_streamer.message import Message

redis = RedisInstance(RedisConfig.from_env("redis_queue_"))
streamer = RedisStreamer(redis_instance=redis, consumer_name="my_service")

streamer.register_channel("my_service_events")
streamer.subscribe("dealer_updated")
streamer.register_handler("dealer_updated", handle_dealer_updated)

# Publish
streamer.send_message("my_service_events", Message(name="event_name", body=payload))

# Consume (blocking)
streamer.persist_consume()
```

---

## Module reference

| Module | Extra | Docs |
|---|---|---|
| `dtpyfw.core` | base | [docs/core/](docs/core/) |
| `dtpyfw.log` | base | [docs/log/](docs/log/) |
| `dtpyfw.api` | `api` | [docs/api/](docs/api/) |
| `dtpyfw.db` | `db` | [docs/db/](docs/db/) |
| `dtpyfw.bucket` | `bucket` | [docs/bucket/](docs/bucket/) |
| `dtpyfw.redis` | `redis` | [docs/redis/](docs/redis/) |
| `dtpyfw.redis_streamer` | `redis_streamer` | [docs/redis_streamer/](docs/redis_streamer/) |
| `dtpyfw.worker` | `worker` | [docs/worker/](docs/worker/) |
| `dtpyfw.kafka` | `kafka` | [docs/kafka/](docs/kafka/) |
| `dtpyfw.opensearch` | `opensearch` | [docs/opensearch/](docs/opensearch/) |
| `dtpyfw.ftp` | `ftp` | [docs/ftp/](docs/ftp/) |
| `dtpyfw.encrypt` | `encrypt` | [docs/encrypt/](docs/encrypt/) |
| `dtpyfw.diagnostics` | base | [docs/diagnostics/](docs/diagnostics/) |

---

## Development

```bash
# Install all dependencies
poetry install -E all

# Run tests
pytest

# Format
black .

# Lint
ruff check . --fix

# Type check
mypy dtpyfw
```

---

## Version history

Current version: **1.0** — see [CHANGELOG.md](CHANGELOG.md) for full history and [UPGRADE.md](UPGRADE.md) for migration guides.

---

## License

DealerTower Python Framework is proprietary software. See [LICENSE](LICENSE) for complete terms and conditions.

---

## Resources

- **Repository**: [github.com/datgate/dtpyfw](https://github.com/datgate/dtpyfw)
- **Issue Tracker**: GitHub Issues

