Metadata-Version: 2.4
Name: daio
Version: 1.2.0
Summary: video and data IO tools for Python
Author: jlab.berlin
License-Expression: MIT
Project-URL: Homepage, https://github.com/danionella/daio
Keywords: python,HDF5,data IO,video IO
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: av>=17
Requires-Dist: h5py>=3
Requires-Dist: hdf5plugin
Provides-Extra: pandas
Requires-Dist: pandas; extra == "pandas"
Dynamic: license-file

![Python Version](https://img.shields.io/badge/python-3.10+-blue)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
![tests](https://github.com/danionella/daio/actions/workflows/test.yml/badge.svg)
[![PyPI - Version](https://img.shields.io/pypi/v/daio)](https://pypi.org/project/daio/)
[![Conda Version](https://img.shields.io/conda/vn/conda-forge/daio.svg)](https://anaconda.org/conda-forge/daio)
![GitHub last commit](https://img.shields.io/github/last-commit/danionella/daio)

# daio
Speed-optimized, frame-accurate video and data IO tools for Python.

Links: [API documentation](http://danionella.github.io/daio), [GitHub repository](https://github.com/danionella/daio)

## Features

#### Video IO
- **Efficient Video Reading**: Use `VideoReader` for fast frame-accurate video reading with support for array-like indexing and iteration.
- **Video Writing**: Write video files easily with `VideoWriter`, supporting multiple codecs and pixel formats.

#### HDF5 File IO
- **Lazy Loading**: Access HDF5 files with a dictionary-like interface using `lazyh5`, loading data only when accessed.
- **Nested Data Support**: Save and load nested dictionaries, including arrays, strings, and JSON-serializable objects.
- **Interactive Exploration**: Explore HDF5 file structures interactively in Jupyter notebooks.

## Installation
- via conda or mamba: `conda install conda-forge::daio`
- if you prefer pip: `pip install daio`
- optional: pandas, to save and load DataFrames (`pip install "daio[pandas]"`)

## Use 

### Video IO

Write video:
```python
from daio.video import VideoReader, VideoWriter
writer = VideoWriter('/path/to/video.mp4', fps=25)
for i in range(20):
    frame = np.random.randint(0,255,size=(720,1280), dtype='uint8')
    writer.write(frame)
writer.close()
```

Read video using speed-optimized array-like indexing or iteration:
```python
reader = VideoReader('/path/to/video.mp4')
frame_7 = reader[7]
first10_frames = reader[:10]
for frame in reader:
    process_frame(frame)
reader.close()
```

Indexing is frame-accurate: `reader[i]` returns the same frame as the i-th frame read sequentially. Frames are located using the container index (mp4/mov/avi without B-frames), or the nominal frame rate (constant frame rate mp4/mov with B-frames). In other cases (e.g. variable frame rate with B-frames, dropped frames, edit lists, or mkv/ts containers), `VideoReader` first scans all packet time stamps (no decoding, but reads the whole file) to build an exact frame index. You can force this with `VideoReader(path, index='scan')`; `reader.index_source` tells you which method is used. Per-frame presentation times (in seconds) are available as `reader.frame_times`.

Every seek is verified (the frame must come from the expected position in the file, decoded time stamps must increase, and decoding must start at a real key frame). Some files cannot be seeked reliably, e.g. avi files with B-frames, files with duplicate time stamps, or intra-refresh streams. For those, `VideoReader` warns and switches to sequential decoding (`reader.seekable` becomes `False`): frames are still returned correctly, but jumping backwards decodes from the start of the file, which can be slow. Transcoding such files (e.g. `ffmpeg -i input.avi -c:v libx264 -pix_fmt yuv420p output.mp4`) makes them fast to seek again.

Note on precision: video encoding is lossy by default (`crf=23`). Grayscale frames are stored in the luma channel of `yuv420p` video, which uses the limited range 16-235, so even lossless settings (`crf=0`) can change gray values by ±1. If you need exact values, store the data in HDF5 instead, or encode RGB frames losslessly with e.g. `VideoWriter(path, codec='libx264rgb', pix_fmt='rgb24', crf=0)`.

You can also use with statements to handle file closure:
```python
with VideoWriter('/path/to/video.mp4', fps=25) as writer:
    for i in range(20):
        frame = np.random.randint(0,255,size=(720,1280), dtype='uint8')
        writer.write(frame)
#or
with VideoReader('/path/to/video.mp4') as reader:
    frame_7 = reader[7]
```

### HDF5 file IO

Lazily load HDF5 with a dict-like interface (contents are only loaded when accessed):
```python
from daio.h5 import lazyh5
h5 = lazyh5('/path/to/datafile.h5')
b_loaded = h5['b']
e_loaded = h5['c']['e']
h5.keys()
```

Create a new HDF5 file (or add items to existing file by setting argument `readonly=False`):
```python
h5 = lazyh5('test.h5')
h5['a'] = 1
h5['b'] = 'hello'
h5['c'] = {} # create subgroup
h5['c']['e'] = [2,3,4]
```

Load entire HDF5-file to dict, or save dict to HDF5-file:
```python
# save dict to HDF5 file:
some_dict = dict(a = 1, b = np.random.randn(3,4,5), c = dict(g='nested'), d = 'some_string')
lazyh5('/path/to/datafile.h5').from_dict(some_dict)
# load dict from HDF5 file:
loaded = lazyh5('/path/to/datafile.h5').to_dict()
```

In Jupyter, you can interactively explore the file structure:

<img width="598" alt="image" src="https://github.com/user-attachments/assets/878d4af6-ce26-4ec3-9ba5-2701fba1c07e">


-----
<details><summary>Old interface (expand this)</summary>
    
```python
from daio.h5 import save_to_h5, load_from_h5
# save dict to HDF5 file:
some_dict = dict(a = 1, b = np.random.randn(3,4,5), c = dict(g='nested'), d = 'some_string')
save_to_h5('/path/to/datafile.h5', some_dict)
# load dict from HDF5 file:
dict_loaded = load_from_h5('/path/to/datafile.h5')
```

</details>
