Metadata-Version: 2.4
Name: vsifile
Version: 0.5.0
Summary: Super simple file reader.
Project-URL: Homepage, https://developmentseed.github.io/vsifile/
Project-URL: Documentation, https://vincentsarago.github.io/vsifile/
Project-URL: Issues, https://github.com/vincentsarago/vsifile/issues
Project-URL: Source, https://github.com/vincentsarago/vsifile
Project-URL: Changelog, https://vincentsarago.github.io/vsifile/release-notes/
Author-email: Vincent Sarago <vincent@developmentseed.com>
License: MIT License
        
        Copyright (c) 2023 Vincent Sarago
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Requires-Python: >=3.11
Requires-Dist: attrs
Requires-Dist: cachetools
Requires-Dist: diskcache
Requires-Dist: httpx
Requires-Dist: obstore<0.9,>=0.5
Requires-Dist: pydantic
Requires-Dist: pydantic-settings
Description-Content-Type: text/markdown

# vsifile

<p align="center">
  <a href="https://github.com/vincentsarago/vsifile/actions?query=workflow%3ACI" target="_blank">
      <img src="https://github.com/vincentsarago/vsifile/workflows/CI/badge.svg" alt="Test">
  </a>
  <a href="https://codecov.io/gh/vincentsarago/vsifile" target="_blank">
      <img src="https://codecov.io/gh/vincentsarago/vsifile/branch/main/graph/badge.svg" alt="Coverage">
  </a>
  <a href="https://pypi.org/project/vsifile" target="_blank">
      <img src="https://img.shields.io/pypi/v/vsifile?color=%2334D058&label=pypi%20package" alt="Package version">
  </a>
  <a href="https://github.com/vincentsarago/vsifile/blob/main/LICENSE" target="_blank">
      <img src="https://img.shields.io/github/license/vincentsarago/vsifile.svg" alt="Downloads">
  </a>
</p>

---

**Documentation**: <a href="https://vincentsarago.github.io/vsifile/" target="_blank">https://vincentsarago.github.io/vsifile/</a>

**Source Code**: <a href="https://github.com/vincentsarago/vsifile" target="_blank">https://github.com/vincentsarago/vsifile</a>

---

## Description

Experiment using Rasterio/GDAL **Python file opener VSI plugin** https://github.com/rasterio/rasterio/pull/2898/files

Future version of rasterio will accept an custom dataset `opener`:

```
opener : callable, optional
        A custom dataset opener which can serve GDAL's virtual
        filesystem machinery via Python file-like objects. The
        underlying file-like object is obtained by calling *opener* with
        (*fp*, *mode*) or (*fp*, *mode* + "b") depending on the format
        driver's native mode. *opener* must return a Python file-like
        object that provides read, seek, tell, and close methods.
```
ref: https://github.com/rasterio/rasterio/blob/d966440c06f3324aca1fa761d490cc780a9f619c/rasterio/__init__.py#L185-L191


## Install

You can install `vsifile` using pip

```bash
python -m pip install -U pip
python -m pip install -U vsifile
```

or install from source:

```bash
git clone https://github.com/vincentsarago/vsifile.git
cd vsifile
python -m pip install -U pip
python -m pip install -e .
```

## Usage

```python
from vsifile import VSIFile, FileReader

src_path = "tests/fixture.cog.tif"

with VSIFile(src_path, "rb") as f:
    assert isinstance(f, FileReader)
    assert hash(f)
    assert "FileReader" in str(f)

    assert not f.closed
    assert f.header_cache
    assert len(f.header) == 32768
    assert f.tell() == 0
    assert f.seekable

    b = f.read(100)
    assert len(b) == 100
    assert f.header[0:100] == b
    assert f.tell() == 100

    _ = f.seek(0)
    assert f.tell() == 0

    _ = f.seek(40000)
    assert f.tell() == 40000

    b = f.read(100)
    assert f.tell() == 40100

    # fetch the same block (should be from LRU cache)
    _ = f.seek(40000)
    b_cache = f.read(100)
    assert f.tell() == 40100
    assert b_cache == b

    b = f.get_byte_ranges([100, 200], [10, 20])
    assert len(b) == 2
    assert len(b[0]) == 10
    assert len(b[1]) == 20
    assert f.tell() == 220
```

#### With Rasterio

```python
import rasterio
from vsifile.rasterio import opener

with rasterio.open("tests/fixtures/cog.tif",  opener=opener) as src:
    ...
```

### Caches Configuration

#### Header Cache

*vsifile* uses [DiskCache](https://grantjenks.com/docs/diskcache/) to create a **persistent** File Header cache (TTL: *Time To Live* cache).
By default the cache will be cleaned up when closing the file handle, you can change this behaviour by setting `VSIFILE_CACHE_DIRECTORY="{your temp directory}"` environment variable.

Settings:

- **VSIFILE_CACHE_DIRECTORY**: Diskcache directory (defaults to `None`)
- **VSIFILE_CACHE_HEADERS_TTL**: Time to Live of each object in the cache, in seconds (defaults to `300`)
- **VSIFILE_CACHE_HEADERS_MAXSIZE**: Maximum size of the cache, in Bytes (defaults to `5120000000`)

#### Block Cache

*vsifile* has a second layer of cache for the `blocks` (non-header read) based on [cachetools](https://cachetools.readthedocs.io/en/stable/index.html#cachetools.TTLCache).

Settings:

- **VSIFILE_CACHE_BLOCKS_TTL**: Time to Live of each object in the cache, in seconds (defaults to `300`)
- **VSIFILE_CACHE_BLOCKS_MAXSIZE**: Maximum size of the cache, in number of items (defaults to `512`)

Note: you can disable cache by setting: **VSIFILE_CACHE_DISABLE=TRUE**

### Other Configurations

- **VSIFILE_INGESTED_BYTES_AT_OPEN**: Bytes ingested when opening a file (header) (defaults to `32768`)

## Contribution & Development

See [CONTRIBUTING.md](https://github.com/vincentsarago/vsifile/blob/main/CONTRIBUTING.md)

## Changes

See [CHANGES.md](https://github.com/vincentsarago/vsifile/blob/main/CHANGES.md).

## License

See [LICENSE](https://github.com/vincentsarago/vsifile/blob/main/LICENSE)
