Metadata-Version: 2.4
Name: lakesoul
Version: 2.0.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Programming Language :: Python :: 3
Requires-Dist: pyarrow>=16,<21
Requires-Dist: numpy
Requires-Dist: protobuf>=5.0
Requires-Dist: typing-extensions>=4.8
Requires-Dist: colorlog>=6
Requires-Dist: torch>=1.11,<2.5 ; extra == 'all'
Requires-Dist: pandas>=1.4 ; extra == 'all'
Requires-Dist: datasets>=2.20 ; extra == 'all'
Requires-Dist: ray>=2.55,<2.56 ; extra == 'all'
Requires-Dist: daft>=0.7.15 ; extra == 'all'
Requires-Dist: pyspark==3.5.8 ; extra == 'all'
Requires-Dist: duckdb>=1.4.0 ; extra == 'all'
Requires-Dist: daft>=0.7.15 ; extra == 'daft'
Requires-Dist: datasets>=2.20 ; extra == 'datasets'
Requires-Dist: duckdb>=1.4.0 ; extra == 'duckdb'
Requires-Dist: pandas>=1.4 ; extra == 'pandas'
Requires-Dist: pyspark==3.5.8 ; extra == 'pyspark'
Requires-Dist: ray>=2.55,<2.56 ; extra == 'ray'
Requires-Dist: torch>=1.11.0,<2.5 ; extra == 'torch'
Provides-Extra: all
Provides-Extra: daft
Provides-Extra: datasets
Provides-Extra: duckdb
Provides-Extra: pandas
Provides-Extra: pyspark
Provides-Extra: ray
Provides-Extra: torch
Summary: Python APIs for using LakeSoul
Keywords: lakesoul,bigdata,ai
Author-email: LakeSoul Team <lakesoul-technical-discuss@lists.lfaidata.foundation>
License: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Bug Tracker, https://github.com/lakesoul-io/LakeSoul/issues
Project-URL: Homepage, https://github.com/lakesoul-io/LakeSoul

# LakeSoul Python Support

## Native Writer

The native writer accepts PyArrow batches, tables, and record batch readers. It
writes data files and returns their metadata; publishing those files to the
LakeSoul metadata service is a separate operation.

```python
import pyarrow as pa

from lakesoul.io import IOConfig, Writer

table = pa.table({"id": [1, 2], "value": ["a", "b"]})
config = IOConfig(
    path="./output",
    schema=table.schema,
    format="parquet",
)

with Writer(config) as writer:
    writer.write(table)

print(writer.result.files)
```

## Ray Writer

Importing `lakesoul.ray` registers a Pythonic write method on Ray datasets. The
target LakeSoul table must already exist. Data files are written by Ray tasks,
then committed to LakeSoul metadata by the driver after every task succeeds.

```python
import ray
import lakesoul.ray

dataset = ray.data.from_items(
    [
        {"id": 1, "value": "a"},
        {"id": 2, "value": "b"},
    ]
)
dataset.write_lakesoul(
    "target_table",
    namespace="default",
    format="parquet",
)
```

