Metadata-Version: 2.5
Name: berg-iceberg
Version: 1.1.0
Summary: A developer and operational management layer for Apache Iceberg.
Requires-Python: <3.15,>=3.13
Requires-Dist: filelock<4,>=3.18
Requires-Dist: psycopg2-binary<3,>=2.9
Requires-Dist: pyarrow<24,>=19
Requires-Dist: pydantic<3,>=2.11
Requires-Dist: pyiceberg[glue,hive,s3fs,sql-postgres]<0.13,>=0.12
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: rich<15,>=14
Requires-Dist: typer<0.28,>=0.27.2
Description-Content-Type: text/markdown

# Berg

[![CI](https://github.com/Berg-Cloud/berg/actions/workflows/ci.yml/badge.svg)](https://github.com/Berg-Cloud/berg/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/berg-iceberg)](https://pypi.org/project/berg-iceberg/)
[![Python](https://img.shields.io/pypi/pyversions/berg-iceberg)](https://pypi.org/project/berg-iceberg/)

Berg is a safe management layer for Apache Iceberg catalogs and table
metadata. It helps teams inspect tables, identify health problems, describe
desired state, review deterministic changes, and execute approved operations
inside their own infrastructure.

Berg does not replace Spark, Flink, Trino, Athena, or another query engine. It
does not move or transform table records. Its focus is controlled metadata and
catalog operations with explicit approval, backend capability checks, durable
execution state, idempotency, audit history, and failure reconciliation.

## Install

With pip:

```bash
python -m pip install berg-iceberg
```

With uv:

```bash
uv tool install berg-iceberg
```

Then:

```bash
berg --version
berg --help
```

## What you can do

```bash
berg connect
berg capabilities
berg scan
berg inspect analytics.events
berg health analytics.events
```

Use `--output json` for automation and CI/CD integrations.

For a metadata change, Berg uses a reviewable plan:

```bash
berg plan analytics.events \
  --set-property write.target-file-size-bytes=536870912 \
  --save events-plan.json

berg apply events-plan.json --dry-run
berg apply events-plan.json --approve --audit-file audit.jsonl
berg execution-status ID --output json
```

Plans are checked against current table state and the selected backend. Berg
rejects unsupported operations, stale plans, unauthorized changes, and unsafe
duplicate execution before mutation.

## Declarative table management

Desired state can be written as a portable `berg.dev/v1` YAML or JSON document:

```bash
berg validate-spec table.yaml
berg diff table.yaml
berg plan --spec table.yaml --save table-plan.json
```

Python users can author the same document without connecting to a catalog:

```python
from berg.sdk import TableDefinition

events = (
    TableDefinition.for_table("analytics.events")
    .with_column("id", "long", required=True)
    .with_column("created_at", "timestamp", required=True)
    .partition_by("created_at", transform="day")
    .with_property("write.target-file-size-bytes", "536870912")
)

events.write("events.yaml")
```

## Supported catalog paths

Berg has compatibility contracts for:

- Iceberg REST Catalog with S3-compatible storage
- Project Nessie
- Apache Polaris
- JDBC/PostgreSQL catalogs
- Hive Metastore
- AWS Glue Catalog with S3

Support is operation- and backend-specific. Run `berg capabilities` and read
the [compatibility matrix](docs/compatibility-matrix.md) before enabling an
operation in automation.

## Customer-local Agent

The Agent runs the same guarded execution kernel inside customer
infrastructure. It receives an approved plan, uses customer credentials, and
writes state and audit records locally. It requires no inbound service and
does not upload raw table data to Berg Cloud.

```bash
berg agent apply plan.json --approve
```

The reference container deployment is in
[docker-compose.agent.yml](docker-compose.agent.yml).

## Local development

The private conformance repository contains disposable Docker environments and
live compatibility contracts. The core repository contains the installable
package and its offline test suite:

```bash
git clone https://github.com/Berg-Cloud/berg.git
cd berg
uv sync --group dev
uv run pytest
```

See the [conformance repository](https://github.com/Berg-Cloud/berg-conformance)
for local catalog stacks. See [local development](docs/local-development.md), the
[capability matrix](docs/capabilities.md), and the
[Python API guide](docs/python-api.md) for details.

The live backend matrix and disposable acceptance environments are maintained
in the private [Berg Conformance repository](https://github.com/Berg-Cloud/berg-conformance).
See the [repository topology](docs/repository-topology.md) for the ownership
and migration rules.

## Safety boundaries

Berg currently supports approved metadata operations such as properties,
compatible schema additions, additive partition fields, certified snapshot
expiration, and guarded orphan-file cleanup. Compaction, arbitrary
destructive changes, and unsupported backend operations remain disabled or
capability-gated.

The product roadmap and release boundaries are documented in
[V0_SCOPE.md](V0_SCOPE.md), [V1_SCOPE.md](V1_SCOPE.md), and
[docs/roadmap.md](docs/roadmap.md).

## License and security

See [SECURITY.md](SECURITY.md) for vulnerability reporting and
[docs/security.md](docs/security.md) for the operational security model.
