Metadata-Version: 2.4
Name: OperonDBS
Version: 0.8.0
Summary: Operon the Database System: Archive, Quality-Control, Organize, Analyze and Release Your Bio-Data
Author-email: hyli360 <lihuanyu2003@gmail.com>
License-Expression: AGPL-3.0-or-later
Project-URL: Repository, https://github.com/HYLi360/Operon
Project-URL: Issues, https://github.com/HYLi360/Operon/issues
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp>=3.9
Requires-Dist: biopython>=1.85
Requires-Dist: paramiko>=3.4
Requires-Dist: PyYAML>=6.0
Requires-Dist: questionary>=2.1
Requires-Dist: requests>=2.32
Requires-Dist: textual>=8.0
Provides-Extra: test
Requires-Dist: coverage>=7.0; extra == "test"
Requires-Dist: cython>=3.0; extra == "test"
Requires-Dist: paramiko>=3.4; extra == "test"
Requires-Dist: pytest>=8.0; extra == "test"
Requires-Dist: pytest-cov>=7.0; extra == "test"
Requires-Dist: pytest-xdist>=3.8; extra == "test"
Requires-Dist: setuptools>=77; extra == "test"
Requires-Dist: textual>=8.0; extra == "test"
Provides-Extra: docs
Requires-Dist: Sphinx<9,>=7.4; extra == "docs"
Requires-Dist: myst-parser<5,>=4.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme<4,>=3.0; extra == "docs"
Requires-Dist: tomli>=2.0; python_version < "3.11" and extra == "docs"
Provides-Extra: dev
Requires-Dist: coverage>=7.0; extra == "dev"
Requires-Dist: cython>=3.0; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=7.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.8; extra == "dev"
Requires-Dist: setuptools>=77; extra == "dev"
Requires-Dist: tomli>=2.0; python_version < "3.11" and extra == "dev"
Requires-Dist: Sphinx<9,>=7.4; extra == "dev"
Requires-Dist: myst-parser<5,>=4.0; extra == "dev"
Requires-Dist: sphinx-rtd-theme<4,>=3.0; extra == "dev"
Dynamic: license-file

# Operon

[![test status](https://github.com/HYLi360/Operon/actions/workflows/test.yml/badge.svg)](https://github.com/HYLi360/Operon/actions/workflows/test.yml) [![codecov](https://codecov.io/gh/HYLi360/Operon/branch/master/graph/badge.svg?token=BC4LD8UPL2)](https://codecov.io/gh/HYLi360/Operon)

A Python-based, **file-based database** designed for large-scale genomic data, used for archiving, quality control, analysis, and deterministic automation.

[点此阅读中文自述文件。](README_ZH.md)

## Features

- **File-based**: A single SQLite file (`operon.sqlite`) serves as the sole writable source of truth; CSV/XLSX are used for controlled imports, TSV reports are used for read-only exchange, and field contracts are defined by a YAML schema
- **NCBI Datasets Adapter**: Offline-first import of JSON/JSONL, ZIP, or unpacked directories; also supports online download of genome packages with automatic archiving
- **Frozen NCBI Taxonomy Coverage**: Versioned YAML profiles are compiled into family/genus denominators with SHA-256 hashes, allowing separate auditing of current metadata and immutable releases, and generating a list of missing samples
- **Streaming Parsing and Built-in QC**: FASTA, FASTQ, GFF3, and protein FASTA files are not loaded entirely into memory; metrics are written to a long table, and decisions are delegated to the versioned YAML profile rule engine; `value_by` allows thresholds to be selected based on classification metrics such as BUSCO auto-lineage
- **Encapsulated External Analysis**: `config/tools.yaml` specifies the launch methods for BLAST/HMMER/BUSCO, artifact types, constrained runtime parameters, version detection, caching, and result write-back; `analyze` executes the entire library or specified categories with a single command
- **Local Control, Remote Storage and Computing**: SQLite, configuration, and provenance are retained locally, while raw large files can reside on a verified SFTP mirror; the execution backend supports local, Slurm, SSH, and remote Slurm environments
- **Universal Executor**: The structured command executor and `import-qc` can integrate with any external tools, such as QUAST, FastQC, fastp, and CheckM2
- **Immutable release**: Dataset snapshots with manifests, checksums, exclusion reports, and provenance; verifiable via `sha256sum -c`

## Dependencies

- Python 3.10+
- Runtime dependencies: `PyYAML`, `requests`, `aiohttp`, `Biopython`, `Paramiko`, `Textual`, and `questionary`; the built-in QC acceleration extension is compiled when the package is built
- Optional extras: `test` (pytest and Cython), `docs` (documentation tooling), and `dev` (all development/build dependencies)

## Install

```bash
# Install the published package
python3 -m venv .venv
source .venv/bin/activate
python -m pip install OperonDBS
```

For an editable checkout, run from the repository root:

```bash
python -m pip install -e '.[dev]'
```

## Documentation

The complete documentation is maintained in [English](docs/en/index.md) and [Chinese](docs/zh/index.md). To build the Sphinx site locally:

```bash
python -m pip install -e '.[docs]'
sphinx-build -W --keep-going -b html docs docs/_build/html
```

Read the Docs uses the repository's `.readthedocs.yaml` configuration and publishes a language-selection page with mirrored `/en/` and `/zh/` documentation trees.
