Metadata-Version: 2.4
Name: seahorse-coral
Version: 0.4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: typing-extensions>=4.0.0
Requires-Dist: pytest>=7.0 ; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0 ; extra == 'dev'
Requires-Dist: black>=22.0 ; extra == 'dev'
Requires-Dist: isort>=5.0 ; extra == 'dev'
Requires-Dist: mypy>=1.0 ; extra == 'dev'
Requires-Dist: ruff>=0.1.0 ; extra == 'dev'
Provides-Extra: dev
Summary: Python client library for SeahorseDB via Coral API server
Author: Seahorse Team
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://docs.seahorse.ai
Project-URL: Repository, https://github.com/seahorse-ai/seahorse

# seahorse-coral

gRPC-native Python client library for SeahorseDB via Coral.

The low-level Python surface is intentionally:
- Arrow-first for tabular reads (`scan`, `search`, `hybrid`)
- typed-model-first for table metadata, ingest/export, and index status results
- explicit about post-processing (`to_pyarrow()`, `to_pandas()`, `to_polars()`, `to_json()`)

Cluster administration is intentionally outside this package. Node inspection, active-set flush,
segment retry/status, rebalance, placement, and runtime tuning remain available through Coral's
operational interfaces and the internal Rust `coral-client`.

Additional docs:
- Build from source: `docs/build.md`
- Release to PyPI: `docs/release.md`
- Advanced usage: `docs/advanced.md`
- Compatibility policy: `docs/compatibility.md`

## Quickstart (Recommended)

This project intentionally supports multiple ways to define a schema (preset / components / builder).
To keep onboarding simple, **we recommend starting with the preset schema** and only moving to Advanced
when you need customization.

### 1) Create a Coral client

```python
import seahorse_coral as sc

coral = sc.Coral("http://localhost:8080")
```

`Coral` and `AsyncCoral` use the same gRPC-native contract as the Rust client.

### 2) Create a table (preset: `id + vector + metadata`)

```python
import seahorse_coral as sc

schema = sc.default_vector_table_schema(
    dim=384,
    # Optional:
    # id_type=sc.ScalarType.STRING,
)

table = coral.create_table("documents", schema=schema)
```

Table mutation and index readiness helpers return typed Python models:

```python
counts = table.indexed_row_count()  # queryable reader-side counts
print(counts.total_row_count)
print(counts.indexed_counts)

table.update_rows("metadata = '{\"source\":\"updated\"}'", where="id = 1")
table.delete_rows(where="id = 2")
```

The preset creates:
- `id`: `INT64` data column by default. SeahorseDB primary-key columns use the legacy composite value format; the SaaS primary path is Arrow `LargeUtf8`/`LARGE_STRING` even though Coral's public scalar API names it `STRING`.
- `vector`: dense vector column
- `metadata`: `STRING` (nullable; store JSON-encoded strings if you want structured metadata)

### (Optional) Schema building (SchemaBuilder)

If you need customization (more columns, segmentation, multiple indexes, etc.), build a schema explicitly.

#### A) Create table with components (no `SchemaBuilder` object)

```python
import seahorse_coral as sc

table = coral.create_table(
    "documents",
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector")],  # List[IndexDefinition]
)
```

Only single-column hash segmentation is supported by the distributed Python SDK.
Value, hierarchical, and composite segmentation are intentionally not exposed.

#### B) `SchemaBuilder` (constructor style)

```python
import seahorse_coral as sc

schema = sc.SchemaBuilder(
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector", space=sc.IndexSpace.COSINE)],
)

table = coral.create_table("documents", schema=schema)
```

#### C) `SchemaBuilder` (fluent / chain style)

```python
import seahorse_coral as sc

schema = (
    sc.SchemaBuilder()
    .int64("id", nullable=False)
    .vector("vector", dim=384)
    .metadata()
    .with_primary_key("id")
    .hnsw("vector", space=sc.IndexSpace.COSINE)
)

table = coral.create_table("documents", schema=schema)
```

### 3) Insert rows

```python
import json

table.insert_rows(
    [
        {"id": 1, "vector": [0.1, 0.2, 0.3], "metadata": json.dumps({"source": "a"})},
        {"id": 2, "vector": [0.2, 0.1, 0.0], "metadata": json.dumps({"source": "b"})},
    ]
)
```

### (Optional) More insert options

Write APIs are explicit by mode.

```python
import seahorse_coral as sc

# 1) JSONL string (each line is a JSON object)
jsonl = (
    '{"id": 4, "vector": [0.4, 0.4, 0.4], "metadata": "{}"}\n'
    '{"id": 5, "vector": [0.5, 0.5, 0.5], "metadata": "{}"}\n'
)
table.insert_jsonl(jsonl)

# 2) Local Parquet file
# - client converts Parquet -> Arrow IPC stream -> gRPC upload stream
table.insert_parquet("./data/documents.parquet", batch_size=8192)

# 3) Single remote Parquet file
# - server reads the object directly
table.insert_parquet(
    sc.s3_file(
        "path/to/documents.parquet",
        bucket="my-bucket",
        access_key="YOUR_ACCESS_KEY",
        secret_key="YOUR_SECRET_KEY",
        region="ap-northeast-2",
    ),
    options=sc.ImportOptions(reader_batch_size=8192),
)

# 4) Multi-file import from S3
request = sc.s3_file(
    ["path/to/a.parquet", "path/to/b.parquet"],
    bucket="my-bucket",
    access_key="YOUR_ACCESS_KEY",
    secret_key="YOUR_SECRET_KEY",
    region="ap-northeast-2",
)
options = sc.ImportOptions(
    format=sc.FileFormat.PARQUET,
    reader_batch_size=8192,  # reader record batch size
    max_concurrent_files=4,  # optional
)
result = table.import_files(request, options=options)
print(result.total_inserted_row_count)

# `import_files()` is strict by default.
# If any file fails, sc.PartialImportError or sc.ImportFilesError is raised
# and the exception carries the same ImportFilesResult via `.result`.

# 5) Arrow IPC stream bytes (advanced)
# - bytes, pyarrow.Table, pyarrow.RecordBatch, and list[RecordBatch] are supported
table.insert_arrow(arrow_ipc_bytes)
```

### UPSERT full rows

`Table.upsert()` and `AsyncTable.upsert()` accept the same Arrow-compatible
values as `insert_arrow()`: Arrow IPC bytes, `pyarrow.Table`,
`pyarrow.RecordBatch`, or a same-schema sequence of record batches.

```python
import pyarrow as pa
import seahorse_coral as sc

upsert_table = coral.create_table(
    "upsert_documents",
    columns=[
        sc.string_column("id", nullable=False),
        sc.vector_column("vector", dim=3),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector")],
)

separator = "\x1e"
rows = pa.table(
    {
        "id": pa.array(
            [f"document{separator}1", f"document{separator}2"],
            type=pa.large_string(),
        ),
        "vector": pa.FixedSizeListArray.from_arrays(
            pa.array([0.1, 0.2, 0.3, 0.3, 0.2, 0.1], type=pa.float32()),
            3,
        ),
        "metadata": pa.array(
            ['{"source":"refresh"}', '{"source":"new"}'],
            type=pa.large_string(),
        ),
    }
)

result = upsert_table.upsert(rows)
print(result.upserted_row_count)
print(result.inserted_row_count)
print(result.replaced_row_count)
```

UPSERT is available only for primary-key tables and every input row must contain
the full table schema. It accepts Arrow input only; JSONL, dictionaries,
Parquet paths, and partial field updates are not supported. `LastWins` applies
only to duplicate primary keys within one Writer apply. One request can be split
into concurrently scheduled applies, so duplicate keys in separate record
batches have no request-wide input-order guarantee. Send each primary key only
once per request when deterministic ordering is required.

Coral uses the same streaming validation, incoming-row segment routing, bounded
dispatch, and partial-failure boundary as BatchInsert. Primary-key lookup and
replacement are local to the destination segment. Moving a primary key to a
different segmentation value does not remove the old segment's row. If a later
slice fails, earlier slices can remain applied and the call returns an error
without partial success counts or rollback.

The SDK does not retry UPSERT automatically and does not provide exactly-once
request semantics. After a timeout or connection loss, the mutation may already
have succeeded. A caller may resend the same full-row payload when logical value
convergence is acceptable, but the inserted/replaced counts, tombstones,
physical rows, and WAL records can differ. During rollout, enable UPSERT traffic
only after every current and failover Writer supports UPSERT replay, and never
roll back to an older Writer after the first UPSERT WAL record. See the
[UPSERT WAL rollout contract](../docs/guide/operations/upsert-wal-rollout.md).

### 4) Search (dense)

```python
# Dense vector search
#
# Note:
# - `index` is the index name, typically the same as the vector column name.
vec = table.index("vector")

result = vec.search([0.1, 0.2, 0.3], top_k=10)

result = vec.search(
    [0.1, 0.2, 0.3],
    top_k=10,
    ef_search=128,
    select="id, metadata, distance",
    where="id > 0",
    read_visibility="checkpoint",
)
```

`read_visibility` defaults to `"latest"`. A partition, implicit whole-segment, or open-ended set
scope includes the complete durable WAL prefix captured for each selected Reader segment; an
explicit bounded set-only scope remains checkpoint-backed because it does not own the open tail.
Use `"checkpoint"` to read only the checkpoint currently served by each Reader and avoid WAL
replay. This is a segment-local visibility boundary, not a table-global atomic snapshot. See the
[compatibility policy](docs/compatibility.md) before enabling explicit `checkpoint` during a
rolling upgrade.

### 5) Consume Arrow-first tabular results

`scan()` and `search()` return `ResultSet`.

```python
result = table.scan(select="id, metadata", limit=100)

# Low-level Arrow-native access
batches = result.to_record_batches()
arrow_table = result.to_pyarrow()

# Explicit convenience conversions
rows = result.to_json()
df = result.to_pandas()
```

For batch vector search, use `ResultSets`.

```python
results = vec.search_batch([[0.1, 0.2, 0.3], [0.3, 0.2, 0.1]], top_k=10)

for result in results:
    print(result.to_pyarrow())
```

Large-result paths are exposed separately.

```python
for batch in table.scan_stream(select="id, metadata"):
    process(batch)

for result in vec.search_batch_stream([[0.1, 0.2, 0.3], [0.3, 0.2, 0.1]], top_k=10):
    process(result.to_pyarrow())
```

### 6) Bootstrap schema from a parquet file

Use `schema_from_parquet()` when you want to start from an existing parquet layout and then
adjust the schema before table creation. A plain string path is read from the client machine, and
remote/object-store sources should be passed as `FileSource`.

```python
import seahorse_coral as sc

schema = coral.schema_from_parquet("./data/documents.parquet")
schema.with_primary_key("id")

table = coral.create_table("documents_from_parquet", schema=schema)
```

### 7) Export or download parquet

`export_parquet()` writes files on the Coral server side. `download_parquet()` and
`download_parquet_stream()` bring the result back to the client process, and local disk writes stay
explicit via `write_to()` or `download_parquet_to()`.

```python
import seahorse_coral as sc

result = table.export_parquet(
    sc.local_directory("/var/lib/coral/exports/documents"),
    where="id > 100",
    mode="single_file",
)
print(result.files)

downloaded = table.download_parquet(limit=1000)
print(downloaded.filename)
downloaded.write_to("./documents-sample.parquet")

table.download_parquet_to("./documents-full.parquet", where="id > 100")
```

### (Optional) Sparse & hybrid search

Sparse/hybrid search requires a table that has a sparse vector column + inverted index.

```python
import seahorse_coral as sc

schema = sc.SchemaBuilder(
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.sparse_vector_column("sparse_emb"),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[
        sc.hnsw_index("vector"),
        sc.inverted_index("sparse_emb"),
    ],
)

table = coral.create_table("documents_hybrid", schema=schema)

# Sparse vector search (BM25 / inverted index)
sparse_query = "1:0.8 5:0.6 12:0.4"
result = table.index("sparse_emb").search_sparse(
    sparse_query,
    top_k=10,
    bm25_k=1.2,
    bm25_b=0.75,
)

# Hybrid search (dense + sparse + fusion)
# - requires dense_column + sparse_column
result = table.hybrid_search(
    dense_column="vector",
    dense_query=[0.1, 0.2, 0.3],
    sparse_column="sparse_emb",
    sparse_query=sparse_query,
    top_k=10,
    options=sc.HybridSearchOptions(
        fusion="rrf",
        rrf_k=60,
        alpha=0.7,
    ),
)
```

## Next steps

- Advanced schema options (segmentation / indexes): [docs/advanced.md](https://github.com/dn-inc/Coral/blob/main/seahorse-coral/docs/advanced.md)
- Build from source / development: [docs/build.md](https://github.com/dn-inc/Coral/blob/main/seahorse-coral/docs/build.md)

