Metadata-Version: 2.4
Name: eyedatahub
Version: 0.5.0
Summary: Versioned, source-term-aware command-line access tool for ophthalmology datasets.
Author-email: Pooya Khosravi <pooya32kh@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/pooyakhosravi/EyeDataHub
Project-URL: Documentation, https://khosravipooya.com/EyeDataHub/
Project-URL: Repository, https://github.com/pooyakhosravi/EyeDataHub
Project-URL: Issues, https://github.com/pooyakhosravi/EyeDataHub/issues
Keywords: ophthalmology,medical imaging,datasets,dataset catalog,data acquisition,reproducibility,licensing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: Pillow>=9.0
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: matplotlib>=3.7
Requires-Dist: rich>=13.0
Requires-Dist: tqdm>=4.65
Requires-Dist: requests>=2.31
Requires-Dist: gdown>=4.7
Requires-Dist: PyYAML>=6.0
Requires-Dist: python-dotenv>=1.0
Provides-Extra: kaggle
Requires-Dist: kaggle>=1.5; extra == "kaggle"
Provides-Extra: huggingface
Requires-Dist: huggingface_hub>=0.19; extra == "huggingface"
Provides-Extra: synapse
Requires-Dist: synapseclient>=4.0; extra == "synapse"
Provides-Extra: full
Requires-Dist: kaggle>=1.5; extra == "full"
Requires-Dist: huggingface_hub>=0.19; extra == "full"
Requires-Dist: synapseclient>=4.0; extra == "full"
Dynamic: license-file

# EyeDataHub

EyeDataHub is a versioned, community-extensible, source-terms- and access-aware
command-line tool backed by a manually curated ophthalmology dataset snapshot.
It helps users find, inspect, cite, preflight, and acquire supported resources
from their official sources.

EyeDataHub does **not** host or redistribute the indexed third-party datasets.
It does not accept source terms for users or determine legal permission,
ethical acceptability, data quality, clinical validity, or scientific
suitability.

[![License: MIT](https://img.shields.io/badge/code-MIT-yellow.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
[![Catalog](https://img.shields.io/badge/catalog-479%20records-brightgreen.svg)](DATASETS.md)
[![Tests](https://img.shields.io/badge/tests-99%20passing-brightgreen.svg)](tests)

## One-minute overview

The current catalog contains 479 manually source-checked records across 18
primary categories. At the catalog cutoff on 2 August 2026:

- 33 use source-hosted, Google Drive, or GitHub routes without platform credentials;
- 375 use platform clients or APIs with the user's corresponding credentials or configuration;
- 71 use manual, controlled, institutional, or author-contact procedures.

These practical groups are mutually exclusive. Separately, the catalog records
the source interaction visible to a user:

- 46 used a source link requiring neither an account nor manual approval;
- 384 required immediate self-service authentication;
- 11 required user click-through;
- 30 required controlled or manual access;
- 8 required author contact.

A publicly viewable source page can still require a credential for its API or
client, so these two classifications are intentionally not interchangeable.

All 479 records were also reviewed for documented reuse, derivation, subset,
version, mirror, component, and cohort-overlap relationships. The reviewed
graph contains 153 directed assertions involving 100 records. The complete
record matrix, edge evidence, unresolved upstream resources, and rejected
false-positive links are in `hub/audit/resource_relationship_*_2026-08-02.*`.

A separate source review resolved a primary reported quantity for 328 records
and retained 392 quantity-evidence rows. Totals are reported only within exact
units because related resources can overlap and different modalities use
different counting units. The record, evidence, unit-summary, and unresolved
files are in `hub/audit/resource_quantity_*_2026-08-02.*`.

The repository searches returned 18,737 records across Dryad, Mendeley Data,
Kaggle, Figshare, and Hugging Face. Human/model-use screening, duplicate
reconciliation, and source review identified 315 eligible repository records;
164 catalog records came from the other documented search routes. Within the
145 Dryad-backed candidates reviewed specifically for reusable model inputs or
targets, 82 were retained and 63 were excluded. The exclusions remain visible
in the dated screening ledger, including 58 nonhuman records and two
analysis-only deposits.

The complete machine-readable exports are `hub/catalog.json` and
`hub/catalog.csv`. Legacy `hub/metadata.*` and
`hub/open_licence_catalogue.json` files are compatibility artifacts, not
complete-catalog exports, and are not used for current manuscript counts.

## Install

Install release 0.5.0 from [PyPI](https://pypi.org/project/eyedatahub/):

```bash
python -m pip install "eyedatahub==0.5.0"
```

EyeDataHub is tested on Python 3.10 through 3.14.

To install the same release directly from its Git tag:

```bash
python -m pip install "git+https://github.com/pooyakhosravi/EyeDataHub.git@v0.5.0"
```

For development:

```bash
git clone https://github.com/pooyakhosravi/EyeDataHub.git
cd EyeDataHub
python -m pip install -e .
```

Optional platform clients are available through `.[kaggle]`,
`.[huggingface]`, `.[synapse]`, or `.[full]`.

## Safe command-line workflow

```bash
# Read-only discovery
eyehub search --modality oct --access anonymous_direct
eyehub search --task segmentation --source-terms standard-no-nc --json

# Read-only record and citation inspection
eyehub show fives --json
eyehub cite fives --type dataset --format bibtex

# Read-only acquisition preflight
eyehub download fives --data-dir ./data --dry-run --json

# Explicit transfer from the represented official source
eyehub download fives --data-dir ./data --json
```

Search, `show`, `cite`, Python queries, JSON, JSON-LD, and the optional MCP
server are read-only. Only an explicit non-dry-run `download` command may start
transfer. Manual, controlled, author-contact, unavailable, and unsupported
routes return structured status and instructions instead of imitating success.

## What source-terms-aware means

Before acquisition, EyeDataHub displays:

- the raw source-stated terms and evidence URL;
- whether the terms appear to apply to data, metadata, code, a publication,
  challenge participation, mixed components, or an unknown scope;
- registration, authentication, token, click-through, manual approval,
  agreement, author-contact, and institutional restrictions;
- route-verification date and loader test scope;
- warnings for unknown, research-only, noncommercial, no-derivatives, or mixed
  terms.

The tool blocks manual authorization routes, never accepts agreements, and
writes a provenance manifest after successful transfer. This behavior is not
legal advice. The descriptive `standard-no-nc` filter means only that the
normalized source label contains no explicit noncommercial clause; it does not
establish permission for a proposed use.

## Python

```python
from pathlib import Path

from eyedatahub.acquisition import acquire_dataset, preflight_dataset
from eyedatahub.datasets.registry import REGISTRY

dataset = REGISTRY.get_dataset("ophthalwechat")
data_dir = Path("./data")

plan = preflight_dataset(dataset, data_dir)       # read-only
result = acquire_dataset(dataset, data_dir)       # explicit transfer request
print(result.status, result.manifest_path)
```

`REGISTRY` is the retained internal Python class name. Public-facing release
artifacts use *catalog* and *snapshot* because EyeDataHub is not a data
repository.

## Credentials and security

Copy [.env.example](.env.example) to a private `.env` file or use each
platform's supported credential mechanism. EyeDataHub records only an
authentication category or credential presence, never credential values.
`.env` and acquired data directories must not be committed.

Supported backends include official Figshare, Zenodo, Mendeley Data, Dryad,
Kaggle, Hugging Face, PhysioNet, Dataverse, Synapse, GitHub, Google Drive, and
source-hosted file links. Support varies by record and is exposed through
`acquisition_support` and `loader_test_scope`.

## Release record

The 0.5.0 release contains the catalog, schema, source-review logs, resource
citations, documentation, and regeneration scripts, but no indexed third-party
dataset files. A version-specific archival DOI can be added when that release
archive is published.

See [CONTRIBUTING.md](CONTRIBUTING.md) for reviewed additions and
[CLAUDE.md](CLAUDE.md) for the canonical agent guide.

## Citation and licences

Use [CITATION.cff](CITATION.cff) for the software citation and cite each
acquired dataset from its official source. EyeDataHub code is MIT licensed;
the catalog metadata are CC BY 4.0. Third-party data remain under their
source-specific terms and access controls.
