Metadata-Version: 2.1
Name: seedee
Version: 0.3.1
Summary: A (de)serialization library for multidimensional arrays
Home-page: https://gitlab.desy.de/fs-sc/seedee
Author: Tim Schoof
Author-email: tim.schoof@desy.de
License: MIT
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE_THIRDPARTY.txt
Requires-Dist: numpy
Provides-Extra: tests
Requires-Dist: pytest ; extra == 'tests'
Requires-Dist: h5py ; extra == 'tests'
Requires-Dist: hdf5plugin ; extra == 'tests'


# seedee

A data format for n-dimensional arrays.

> This project is in an early stage of development and should be used with caution


## Installation

### Python

Pre-built Python wheels can be installed via `pip`:

```
python3 -m pip install seedee
```

To build a wheel yourself, e.g., if your platform is not supported by the
pre-built wheels, download the latest release and extract it. Inside the source
directory, run:

```
$ cmake -S . -B build -DBUILD_PYTHON=ON -DCMAKE_BUILD_TYPE=Release
$ cmake --build build --target python-wheel --verbose
```

Building requires `cmake` version  `3.13` or newer.

If everything works, the wheel is created in the `build/python/dist/`
subdirectory and can be installed via `pip`.

### C/C++

To build and install the C/C++ libraries, run:
```
$ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=path/to/install/dir
$ cmake --build build --target seedee --verbose
$ cmake --install build
```

Building requires `cmake` version  `3.13` or newer.


## Usage example

### Python

```python
import numpy as np
import seedee

# Create a numpy array of some dimension, shape, and dtype
array = np.arange(48).reshape((6, 8)).astype("uint16")

# Serialize the array using the bitshuffle+lz4 compression method
buffer, meta = seedee.serialize(array, seedee.COMPRESSION_BITSHUFFLE)
# buffer is a 1D numpy array of type uint8
# meta is a string

# Send buffer and meta to a remote process using any preferred method,
# e.g., via a ZMQ multi-message
...

# Deserialize the received buffer and meta into a numpy array
new_array = seedee.deserialize(buffer, meta)

# The resulting array is equal to the original array
assert np.all(array == new_array)
```

### C/C++

See `test` directory for some examples. Note that the C/C++ interface will
likely change in the future.

## Limitations

Only arrays up to 1GB are supported.

## Development

Use the following commands to compile `seedee` and run the tests:
```
cmake -S . -B build -DBUILD_PYTHON=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --target all --verbose
ctest --test-dir build -T test --output-on-failure
```

## Contribution

Please feel free to open issues or pull requests.

## Acknowledgments

`seedee` depends on multiple open source libraries that are included in this
repository under the `thirdparty` directory for convenience. The licenses of
these dependencies can be found in the corresponding subdirectories and in
LICENSE_THIRDPARTY.txt.

The chunk serializer implementations are based on the corresponding HDF5 filter
plugins which are subject to their own licenses:
* LZ4 filter plugin: https://github.com/nexusformat/HDF5-External-Filter-Plugins/blob/master/LZ4/COPYING
* bitshuffle filter plugin: https://github.com/kiyo-masui/bitshuffle/blob/master/LICENSE
* zlib filter plugin: https://github.com/HDFGroup/hdf5/blob/develop/COPYING
