Metadata-Version: 2.4
Name: dandifs
Version: 0.1.0
Summary: A fsspec filesystem for the DANDI archive
Author-email: Yael Balbastre <y.balbastre@ucl.ac.uk>
Maintainer-email: Yael Balbastre <y.balbastre@ucl.ac.uk>
License: MIT License
        
        Copyright (c) 2025 Yaël Balbastre
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/balbasty/dandifs
Project-URL: Issues, https://github.com/balbasty/dandifs/issues
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fsspec[http]>=2022.1.0
Requires-Dist: aiohttp
Requires-Dist: typing_extensions
Provides-Extra: auth
Requires-Dist: keyring!=23.9.0; extra == "auth"
Requires-Dist: keyrings.alt; extra == "auth"
Requires-Dist: pycryptodomex; extra == "auth"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-asyncio; extra == "test"
Requires-Dist: aioresponses; extra == "test"
Requires-Dist: aiohttp<3.14; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff==0.16.5; extra == "lint"
Requires-Dist: codespell; extra == "lint"
Dynamic: license-file

# dandifs

An [`fsspec`](https://filesystem-spec.readthedocs.io) filesystem for the
[DANDI archive](https://dandiarchive.org).

`dandifs` resolves `dandi://` URLs to the bytes behind DANDI assets, including
files that live **inside a Zarr asset**. It is async-first (built on
`aiohttp`, via fsspec's `AsyncFileSystem`), has a small dependency footprint,
and does **not** depend on `dandi` or `dandi-schema`.

## Install

```bash
pip install dandifs
# optional: keyring-based credentials for private/embargoed dandisets
pip install "dandifs[auth]"
```

## URL format

```
dandi://<instance>/<dandiset>[@<version>]/<path>
```

- `<instance>` — a registered instance name (`dandi`, `dandi-staging`,
  `ember`, `ember-sandbox`, ...) or any DANDI-schema server (see *Instances*).
- `<dandiset>` — the six-digit dandiset identifier, e.g. `000026`.
- `@<version>` — optional; defaults to the most recent published version, or
  the draft version if none is published.
- `<path>` — a path within the dandiset. It may descend **into** a Zarr asset:
  in `.../image.zarr/0/0/0`, only `.../image.zarr` is a registered asset and
  `0/0/0` is a key inside that Zarr's store.

## Public surface

The **only** public API is the filesystem class, `DandiFileSystem`.
Everything else is internal.

## Usage

### Via the registered protocol

```python
import fsspec, json

with fsspec.open(
    "dandi://dandi/000026/rawdata/sub-I38/ses-MRI/anat/"
    "sub-I38_ses-MRI-echo-4_flip-4_VFA.json"
) as f:
    info = json.load(f)
```

### Bound to a dandiset

```python
from dandifs import DandiFileSystem

fs = DandiFileSystem("000026")            # bind to a dandiset (+ version)
fs.ls("rawdata")                          # browse
fs.glob("**/anat/*.json")                 # glob assets
with fs.open("rawdata/sub-I38/.../VFA.json") as f:
    data = f.read()
```

### Files inside a Zarr asset

```python
fs = DandiFileSystem("000108")

fs.ls("path/to/image.zarr")               # list entries inside the Zarr
fs.info("path/to/image.zarr/0/0/0")       # stat a chunk
with fs.open("path/to/image.zarr/0/0/0", "rb") as f:
    chunk = f.read()                      # read chunk bytes
```

This also works transparently with libraries that open a store through fsspec,
e.g. `zarr.open("dandi://dandi/<id>/path/to/image.zarr")`.

### Async

`DandiFileSystem` is a real async filesystem. Following fsspec's
[`AsyncFileSystem`](https://filesystem-spec.readthedocs.io/en/latest/async.html)
convention, the asynchronous API **is** the set of underscore-prefixed
coroutine methods (`_ls`, `_info`, `_cat_file`, `_cat`, `_exists`, `_glob`,
`_get`, ...) — the same pattern used by `s3fs`, `gcsfs`, and every other
async fsspec backend. The public, non-underscore names (`ls`, `cat_file`,
...) are the *synchronous* wrappers fsspec generates from those coroutines.
Pass `asynchronous=True` and `await` the coroutines:

```python
fs = DandiFileSystem("000026", asynchronous=True)
entries = await fs._ls("rawdata")
data = await fs._cat_file("rawdata/sub-I38/.../VFA.json")
```

The synchronous API is generated from these coroutines by fsspec and runs on
fsspec's shared background event loop, so it also works inside a running loop
(e.g. Jupyter).

## Authentication

Public dandisets need **no** authentication. For private or embargoed
resources, a token is resolved **lazily** — only after a request returns
`401` — in this order:

1. an explicit `token=` argument to `DandiFileSystem`;
2. the `DANDI_API_KEY` environment variable;
3. a per-instance `<INSTANCE>_API_KEY` variable (e.g. `EMBER_API_KEY`);
4. the system keyring (only if the optional `auth` extra is installed).

```python
fs = DandiFileSystem("000026", token="…")          # explicit
# or: export DANDI_API_KEY=…   (recommended)
```

## Instances

Registered instances: `dandi`, `dandi-staging`, `ember`, `ember-sandbox`, and
`dandi-api-local-docker-tests`. You can also point at any DANDI-schema server:

```python
# by API URL
DandiFileSystem("000001", instance="https://api.my-dandi.org/api")
# by a bare server URL — discovered lazily via the server's /info/ endpoint
DandiFileSystem("000001", instance="https://my-dandi.org")
```

## Licensing

`dandifs` is released under the MIT License (see `LICENSE`). Portions of this
project are derived from [dandi-cli](https://github.com/dandi/dandi-cli)
(Apache-2.0); see the per-file `NOTICE` headers and `OTHER_LICENSES`.
