Metadata-Version: 2.4
Name: velocity-python
Version: 0.1.134
Summary: A rapid application development library for interfacing with data storage
Author-email: Velocity Team <info@codeclubs.org>
License-Expression: MIT
Project-URL: Homepage, https://codeclubs.org/projects/velocity
Project-URL: Documentation, https://codeclubs.org/projects/velocity
Project-URL: Bug Tracker, https://codeclubs.org/projects/velocity
Keywords: database,orm,sql,rapid-development,data-storage
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: sqlparse>=0.5.0
Provides-Extra: aws
Requires-Dist: boto3>=1.35.0; extra == "aws"
Requires-Dist: requests>=2.32.0; extra == "aws"
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1.0; extra == "excel"
Provides-Extra: parquet
Requires-Dist: pyarrow>=15.0.0; extra == "parquet"
Provides-Extra: ods
Requires-Dist: odfpy>=1.4.0; extra == "ods"
Provides-Extra: templates
Requires-Dist: jinja2>=3.1.0; extra == "templates"
Provides-Extra: pdf
Requires-Dist: weasyprint>=60.0; extra == "pdf"
Provides-Extra: http
Requires-Dist: requests>=2.32.0; extra == "http"
Provides-Extra: mysql
Requires-Dist: mysql-connector-python>=9.0.0; extra == "mysql"
Provides-Extra: redis
Requires-Dist: redis>=5.0.0; extra == "redis"
Provides-Extra: mongodb
Requires-Dist: pymongo>=4.6.0; extra == "mongodb"
Provides-Extra: elasticsearch
Requires-Dist: elasticsearch>=8.13.0; extra == "elasticsearch"
Provides-Extra: cassandra
Requires-Dist: cassandra-driver>=3.29.0; extra == "cassandra"
Provides-Extra: couchbase
Requires-Dist: couchbase>=4.2.0; extra == "couchbase"
Provides-Extra: sqlserver
Requires-Dist: python-tds>=1.15.0; extra == "sqlserver"
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.2.0; extra == "postgres"
Provides-Extra: payment
Requires-Dist: stripe<17,>=15.0.0; extra == "payment"
Requires-Dist: braintree<5,>=4.30.0; extra == "payment"
Requires-Dist: xmltodict<2,>=0.13.0; extra == "payment"
Provides-Extra: plaid
Requires-Dist: ecdsa>=0.19.0; extra == "plaid"
Provides-Extra: all
Requires-Dist: velocity-python[aws,excel,http,payment,pdf,plaid,postgres,templates]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov>=6.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=1.0.0; extra == "dev"
Requires-Dist: black>=24.0.0; extra == "dev"
Requires-Dist: flake8>=7.0.0; extra == "dev"
Requires-Dist: mypy>=1.10.0; extra == "dev"
Requires-Dist: pre-commit>=4.0.0; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=8.0.0; extra == "test"
Requires-Dist: pytest-cov>=6.0.0; extra == "test"
Requires-Dist: pytest-mock>=3.14.0; extra == "test"
Requires-Dist: pytest-asyncio>=1.0.0; extra == "test"
Requires-Dist: psycopg[binary]>=3.2.0; extra == "test"
Requires-Dist: boto3>=1.35.0; extra == "test"
Requires-Dist: requests>=2.32.0; extra == "test"
Requires-Dist: stripe<17,>=15.0.0; extra == "test"
Requires-Dist: braintree<5,>=4.30.0; extra == "test"
Requires-Dist: xmltodict<2,>=0.13.0; extra == "test"
Requires-Dist: authorizenet>=1.1.3; extra == "test"
Requires-Dist: ecdsa>=0.19.0; extra == "test"
Requires-Dist: openpyxl>=3.1.0; extra == "test"
Requires-Dist: redis>=5.0.0; extra == "test"
Requires-Dist: fakeredis>=2.23.0; extra == "test"
Requires-Dist: pymongo>=4.6.0; extra == "test"
Requires-Dist: mongomock>=4.1.0; extra == "test"
Requires-Dist: moto[dynamodb]>=5.0.0; extra == "test"
Provides-Extra: docs
Requires-Dist: sphinx>=8.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=3.0.0; extra == "docs"
Dynamic: license-file

# Velocity-Python

A rapid application development library for Python that eliminates boilerplate between your code and your database. Write business logic, not SQL plumbing.

```python
@engine.transaction
def create_order(tx, customer_email, items):
    customer = tx.table("customers").find({"email": customer_email})
    order = tx.table("orders").insert({
        "customer_id": customer["sys_id"],
        "status": "pending",
        "total": sum(i["price"] for i in items),
    })
    tx.table("order_items").insert_many([
        {"order_id": order["sys_id"], "product": i["name"], "price": i["price"]}
        for i in items
    ])
    return order
```

No connection management, no cursor juggling, no commit/rollback boilerplate. Velocity handles it all.

[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT)

---

## Why Velocity?

Most Python database libraries fall into two camps:

1. **Heavy ORMs** (SQLAlchemy, Django ORM) — powerful but complex. You write Python classes that map to tables, manage sessions, deal with migration frameworks, and learn a large API surface before writing your first query.

2. **Raw drivers** (psycopg, sqlite3) — full control, but you're writing SQL strings, managing connections, handling cursors, serializing parameters, and building your own transaction/error-handling patterns from scratch.

Velocity occupies the middle ground: **a thin, opinionated layer** that gives you the convenience of an ORM with the transparency of raw SQL. Tables are just names. Rows are just dicts. Transactions are just context managers. You don't define models — Velocity discovers your schema at runtime and adapts to it.

### Design Principles

| Principle | What It Means |
|-----------|---------------|
| **Convention over configuration** | Sensible defaults everywhere. Override only what you need. |
| **Dicts in, dicts out** | No custom model classes to learn. Rows are dictionaries. |
| **Transaction-scoped** | Every operation runs inside an explicit transaction. No surprise autocommit. |
| **Auto-schema** | Tables and columns are created on the fly in development. Locked down in production. |
| **Driver-agnostic** | PostgreSQL (primary), MySQL, SQLite, SQL Server — same API surface. |
| **Lambda-native** | Connection pooling, warm-start reuse, and SQS batch handling built in. |

---

## Installation

```bash
# Core (no database driver — useful for testing or SQLite)
pip install velocity-python

# PostgreSQL (recommended)
pip install velocity-python[postgres]

# With AWS Lambda support
pip install velocity-python[postgres,aws]

# Everything
pip install velocity-python[all]
```

### Available Extras

| Extra | Packages | Use Case |
|-------|----------|----------|
| `postgres` | `psycopg[binary]>=3.2.0` | PostgreSQL connections |
| `aws` | `boto3`, `requests` | Lambda handlers, SQS, Amplify |
| `excel` | `openpyxl` | Excel export |
| `parquet` | `pyarrow` | Parquet export |
| `ods` | `odfpy` | OpenDocument spreadsheet export |
| `templates` | `jinja2` | Template rendering |
| `pdf` | `weasyprint` | HTML → PDF rendering (native Pango/Cairo from the OS or a layer) |
| `http` | `requests` | HTTP utilities |
| `payment` | `stripe`, `braintree`, `xmltodict` | Stripe and Braintree adapters (Authorize.Net additionally needs `authorizenet`, installed separately — see `docs/payment.md`) |
| `mysql` | `mysql-connector-python` | MySQL connections |
| `sqlserver` | `python-tds` | SQL Server connections |
| `redis`, `mongodb`, `elasticsearch`, `cassandra`, `couchbase` | the matching client | NoSQL adapters (`docs/nosql.md`) |
| `all` | `postgres`, `aws`, `excel`, `templates`, `http`, `payment`, `pdf` | Typical full install |

**Requires Python 3.9+** and uses **psycopg v3** (not psycopg2) for PostgreSQL.

---

## Quick Start

### 1. Connect

```python
from velocity.db.servers.postgres import initialize

# From environment variables (DBHost, DBDatabase, DBUser, DBPassword)
engine = initialize()

# Or explicit config
engine = initialize(config={
    "host": "localhost",
    "dbname": "myapp",
    "user": "postgres",
    "password": "secret",
})
```

### 2. Use Transactions

```python
# As a decorator (recommended for Lambda handlers)
@engine.transaction
def get_active_users(tx):
    return tx.table("users").select(where={"active": True}).all()

# As a context manager
with engine.transaction() as tx:
    tx.table("users").insert({"name": "Alice", "email": "alice@example.com"})
```

### 3. CRUD Operations

```python
@engine.transaction
def demo(tx):
    users = tx.table("users")

    # Insert
    row = users.insert({"name": "Bob", "email": "bob@example.com"})

    # Read
    user = users.row(row["sys_id"])                    # by primary key
    user = users.find({"email": "bob@example.com"})    # by lookup

    # Update
    user["name"] = "Robert"                            # immediate write-through

    # Delete
    users.delete({"sys_id": row["sys_id"]})
```

### 4. Bulk Operations

```python
@engine.transaction
def import_customers(tx, records):
    tx.table("customers").insert_many(records)              # multi-row INSERT
    tx.table("customers").upsert_many(records, pk="email")  # INSERT ... ON CONFLICT UPDATE
```

---

## Documentation

Full documentation is included in the `docs/` directory of the source distribution.
What changed in each release, and what callers must do before upgrading, is in
[`CHANGELOG.md`](CHANGELOG.md) (Breaking / Deploy ordering / Added / Fixed per version).

| Guide | File | Description |
|-------|------|-------------|
| Database ORM | `docs/database.md` | Connections, `DB_TUNNEL`, transactions, tables, rows, results, queries, purge, schema management |
| Schema Management | `docs/schema-management.md` | Runtime auto-heal, declarative `ensure_schema()`, schema locking, conflict handling |
| Performance & Optimization | `docs/performance.md` | Connection pooling, batch operations, query caching, prepared statements, N+1 prevention, observability |
| Async Support | `docs/async.md` | AsyncTransaction, AsyncTable, AsyncResult, parallel queries with gather() |
| Configuration Reference | `docs/configuration.md` | Every environment variable and SSM config key velocity reads, `ssm_config`, S3 buckets, engine options |
| AWS Lambda Handlers | `docs/aws-handlers.md` | LambdaHandler, SqsHandler, auth modes, per-record transactions, DataService client boundary, async jobs (JobHandler + job result stores), failure events |
| AWS Auth Modes | `docs/AWS_HANDLERS_AUTH.md` | Cognito auth, public endpoints, webhooks |
| Deployment | `docs/deploy.md` | `velocity.aws.deploy`: least-privilege IAM, function settings, layers, SQS queue reconcile |
| Cross-database Mirroring | `docs/mirroring.md` | Keep a table identical in two databases (demo ↔ production) by natural key; rwx hooks, FK translation, reconcile |
| Operations Dashboard | `docs/operations.md` | What the automation is doing: schedules, jobs, failures, stuck work, with drill-down (`velocity.aws.ops` + velocity-react `OpsDashboard`) |
| NoSQL Adapters | `docs/nosql.md` | Redis, DynamoDB, MongoDB, Elasticsearch, Cassandra, Couchbase behind one capability-honest contract |
| Payment Processing | `docs/payment.md` | Stripe, Braintree and Authorize.Net adapters, processor mirrors, payment lifecycle |
| Utilities | `docs/utilities.md` | Excel/PDF export, data conversion, formatting, timers, email parsing and SMTP sending |
| Testing Guide | `docs/TESTING.md` | Running tests, markers, coverage |
| Security | `docs/SECURITY.md` | Pre-commit hooks, credential scanning |
| Changelog | `CHANGELOG.md` | Per-release breaking changes, deploy ordering, additions and fixes |
| Improvement Backlog | `docs/IMPROVEMENT_BACKLOG.md` | Proposed and deferred library improvements |

---

## Architecture

```
Engine (singleton — survives Lambda warm starts)
├── ConnectionPool (thread-safe, configurable min/max)
└── Transaction (one per request, borrows from pool)
     ├── Table (CRUD, batch ops, schema management)
     │    ├── Row (dict-like, lazy-cache, write-through, batch_update())
     │    └── Result (streaming cursor iteration, transforms)
     ├── View (create, grant, ensure)
     └── Sequence (create, next, current, configure)
```

Transactions auto-commit on success, auto-rollback on exception. Connections are returned to the pool (or discarded on error). The Engine persists across Lambda invocations, so the pool stays warm.

---

## Multi-Database Support

| Database | Driver | Status |
|----------|--------|--------|
| PostgreSQL | `psycopg[binary]>=3.2.0` | Primary, fully tested |
| MySQL | `mysql-connector-python` | Supported |
| SQLite | `sqlite3` (stdlib) | Supported |
| SQL Server | `python-tds` | Supported |

```python
# PostgreSQL
from velocity.db.servers.postgres import initialize
engine = initialize()

# MySQL
from velocity.db.servers.mysql import initialize
engine = initialize()

# SQLite
from velocity.db.servers.sqlite import initialize
engine = initialize(config={"database": "myapp.db"})

# SQL Server
from velocity.db.servers.sqlserver import initialize
engine = initialize()
```

---

## Project Structure

```
velocity-python/
├── src/velocity/
│   ├── cache.py                  # Process-level read cache (cache_ttl / memo)
│   ├── logging.py                # Logging setup (JSON in Lambda, text elsewhere)
│   ├── db/
│   │   ├── core/
│   │   │   ├── engine.py         # Engine, ConnectionPool, @engine.transaction
│   │   │   ├── lazy_engine.py    # LazyEngine (build on first use)
│   │   │   ├── transaction.py    # Transaction, query timing, caching, on_commit
│   │   │   ├── table.py          # Table CRUD, batch ops, purge, schema
│   │   │   ├── row.py            # Row (dict-like, write-through, batch_update())
│   │   │   ├── result.py         # Result (cursor wrapper, transforms)
│   │   │   ├── column.py         # Column helpers
│   │   │   ├── database.py       # Database-level operations
│   │   │   ├── jsonproxy.py      # Live list/dict values for JSON columns
│   │   │   ├── async_support.py  # Async versions of core classes
│   │   │   ├── view.py           # View management
│   │   │   ├── sequence.py       # Sequence management
│   │   │   └── decorators.py     # @create_missing, @return_default, retries
│   │   ├── servers/
│   │   │   ├── postgres/         # PostgreSQL dialect + initializer (primary)
│   │   │   ├── mysql/            # MySQL dialect
│   │   │   ├── sqlite/           # SQLite dialect
│   │   │   ├── sqlserver/        # SQL Server dialect
│   │   │   └── tablehelper.py    # Where/column/pointer-join compiler
│   │   ├── fragments.py          # Literal / @@ token handling
│   │   ├── maintenance.py        # DatabaseMaintenance (clone, snapshot, prune)
│   │   ├── migrations.py         # `velocity` CLI: migration runner
│   │   ├── tunnel.py             # DB_TUNNEL support
│   │   └── exceptions.py         # Db* exception hierarchy
│   ├── aws/
│   │   ├── handlers/             # LambdaHandler, SqsHandler, context, mixins
│   │   │   └── mixins/           # DataService, client-input guard, jobs, sessions, web
│   │   ├── capabilities/         # Per-site capability registry and provisioning
│   │   ├── assets/               # Asset store: indexing, references, usage
│   │   ├── deploy/               # Platform-neutral deployment engine
│   │   ├── ssm_config.py         # SSM-backed config (project, then /Global)
│   │   ├── s3.py                 # Presigned URLs, configured buckets
│   │   ├── failures.py           # Failure events (SNS)
│   │   ├── feature_flags.py      # sys_feature_flags cache + mirror
│   │   ├── mirror.py             # Cross-database table mirroring
│   │   ├── job_lifecycle.py      # Job status tracking
│   │   ├── job_results.py        # Job result stores
│   │   ├── dirty_pipeline.py     # sys_dirty / on_dirty pipeline
│   │   ├── ops.py                # Operations dashboard data
│   │   ├── email.py / sms.py     # SES email, SMS
│   │   └── amplify.py, amplify_build.py, blobs.py, facets.py
│   ├── nosql/                    # NoSQL store contract + adapters
│   ├── payment/
│   │   ├── base_adapter.py       # Abstract payment interface
│   │   ├── stripe_adapter.py     # Stripe
│   │   ├── braintree_adapter.py  # Braintree
│   │   ├── authorizenet_adapter.py # Authorize.Net (CIM)
│   │   ├── *_mirror.py           # Processor mirror tables
│   │   └── money.py, charge_rules.py, redact.py
│   └── misc/                     # conv (iconv/oconv), export, format, mail, pdf, timer, ...
├── tests/                        # Unit tests (make test-unit)
├── docs/                         # Detailed documentation
└── pyproject.toml
```

---

## Development

```bash
# Install with dev dependencies
pip install -e ".[dev,test,postgres]"

# Run the unit suite (no database needed)
make test-unit

# Or directly
pytest

# Run with coverage
pytest --cov=velocity --cov-report=html

# Run specific test file
pytest tests/test_connection_pool.py -v
```

---

## License

MIT
