Metadata-Version: 2.5
Name: ChemVault
Version: 0.14.7
Summary: A Cheminformatics Application for Traceable Target-Centered Bioactivity Dataset Construction and Modelability Assessment
Author: Ana C. Murrieta, Diego Castro-Flores, Fernanda I. Saldívar-González
Author-email: "Flavio F. Contreras-Torres" <contreras.flavio@tec.mx>
License-Expression: LGPL-3.0-or-later
License-File: COPYING
License-File: COPYING.LESSER
License-File: LICENSE
Keywords: PubChem Bioassays,bioactivity data curation,chemical structure harmonization,modelability assessment
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Chemistry
Requires-Python: >=3.12
Requires-Dist: fastapi>=0.138.1
Requires-Dist: harmonsmile==0.3.4
Requires-Dist: httpx2>=2.5.0
Requires-Dist: molraptor==0.4.1
Requires-Dist: numpy>=2.5.0
Requires-Dist: pandas>=3.0.3
Requires-Dist: pillow>=12.2.0
Requires-Dist: requests>=2.34.2
Requires-Dist: streamlit>=1.58.0
Requires-Dist: uvicorn==0.49.0
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Description-Content-Type: text/markdown

# CHEMVAULT

**Traceable Target-Centered Bioactivity Dataset Construction and Modelability Assessment**

[![License: LGPL v3](https://img.shields.io/badge/License-LGPL_v3-blue.svg)](LICENSE)

---

## What is CHEMVAULT?

**CHEMVAULT** is a molecular dataset management and curation
application for creating, organizing, tracing, and exporting molecular and
structured activity datasets from PubChem or CSV sources.

It provides traceable workflows for target-based dataset construction, SMILES
harmonization, structured activity filtering, preservation of PubChem activity
labels, structure consolidation, and dataset modelability assessment.

CHEMVAULT can:

- build SQLite databases from one PubChem protein target, either resolved from
  a gene symbol and one supported organism or supplied directly as a UniProt
  accession, or from CSV files;
- inspect, filter, and export molecular and structured activity data;
- harmonize PubChem-derived molecular structures through HARMONSMILE;
- preserve PubChem `Active` and `Inactive` activity labels;
- consolidate repeated observations by harmonized molecular structure;
- evaluate structure-label neighborhood consistency with the
  **MODELABILITY INDEX** and characterize scaffold-level structural context;
- preserve target identity, source provenance, persisted job state,
  AID-level PubChem source correspondence, reusable fingerprint artifacts,
  and downloadable analysis outputs.

The MODELABILITY INDEX workflow consumes fingerprints
generated by [MOLRAPTOR](https://pypi.org/project/molraptor/) and evaluates
the organization of the labeled dataset. CHEMVAULT complements this analysis
with Bemis-Murcko structural context to distinguish local fingerprint-space
concordance from scaffold-level label organization.

CHEMVAULT is not a machine-learning training application. It does not fit
predictive models, define activity thresholds, or implement molecular
fingerprint algorithms.

CHEMVAULT is developed and maintained by the
[NanoBiostructures Research Group](https://github.com/NanoBiostructuresRG-lab).

---

## Core workflow

```text
CSV input
    |
    v
SQLite table
    |
    v
Explore, refine, derive tables, and export CSV


Gene symbol + supported organism -> reviewed UniProt accession
                                      |
Direct UniProt accession -------------+
                                      |
                                      v
                            PubChem protein search
                                      |
                                      v
                  Molecular, assay, and structured activity tables
                                      |
                                      v
                     SMILES HARMONIZED (HARMONSMILE)
                                      |
                                      v
                       Structured activity filtering
                                      |
                                      v
                 ACTIVITY LABELS and structure consolidation
                                      |
                                      v
                 MODELABILITY INDEX (MOLRAPTOR fingerprints)
                                      |
                                      v
             Nearest-neighbor CSV, analysis JSON, and fingerprint .npz exports
```

CHEMVAULT keeps source tables available and records derived-table provenance so
that curated and analytical outputs remain traceable to their inputs.

For PubChem protein-search builds, CHEMVAULT also preserves the same-run source
enumeration of `(protein, AID)` identities and evaluates whether those
identities correspond to the terminal persisted `(protein, AID)` population
or to an explicit row-level target exclusion. This is an AID-population correspondence
check; it does not certify completeness of source rows, individual CIDs,
structured activity records, molecular properties, or other downstream
content.

---

## Who is it for?

CHEMVAULT is intended for researchers in chemistry, biochemistry,
cheminformatics, chemical biology, and related fields who need a visual
workflow for preparing molecular datasets with preserved provenance,
traceability, and reproducibility.

---

## Prerequisites

Install:

- [Git](https://git-scm.com/)
- [Conda](https://docs.conda.io/en/latest/)

The supported release environment uses Python 3.12.

---

## Installation

> **Windows:** Anaconda Prompt, PowerShell, or the VS Code integrated terminal
> can be used when Conda is configured.
>
> **macOS / Linux:** use the system terminal.

CHEMVAULT declares its compatible runtime dependencies in `pyproject.toml`.
Create and activate a Python 3.12 environment before installing it:

```bash
conda create -n chemvault_env python=3.12
conda activate chemvault_env
```

### Install a published distribution from PyPI

For a CHEMVAULT release that is available from PyPI, install the published
distribution with:

```bash
python -m pip install ChemVault
```

### Install from the source repository

Clone the repository and, when required, select the release tag or development
branch that you intend to install:

```bash
git clone https://github.com/NanoBiostructuresRG-lab/chemvault.git
cd chemvault
```

Install CHEMVAULT and its declared dependencies from the selected checkout:

```bash
python -m pip install .
```

For either installation method, launch the installed application with:

```bash
chemvault
```

Use `chemvault --help` for CLI help or `chemvault --version` to display the
installed version.

### Development installation and execution

For development directly from a source checkout, install CHEMVAULT in editable
mode and run the Streamlit application from the repository root:

```bash
python -m pip install -e .
streamlit run app.py
```

Open `http://localhost:8501` in your browser.

> Do not install HARMONSMILE or MOLRAPTOR separately for normal
> CHEMVAULT use. The compatible versions are defined in `pyproject.toml`.

### Documentation

The documentation source provides the [Usage guide](docs/usage.md),
[API reference](docs/api.md), and [Changelog](CHANGELOG.md). To install the
documentation dependencies and serve the site locally from a source checkout:

```bash
python -m pip install -e ".[docs]"
python -m mkdocs serve
```

These commands serve the documentation locally and do not indicate that the
GitHub Pages site has been deployed.

### Updating a development checkout

```bash
cd path/to/chemvault
git fetch origin
git pull --ff-only
conda activate chemvault_env
python -m pip install -e .
```

---

## Execution modes

CHEMVAULT supports two execution modes.

### Local mode

Local mode is the default and recommended mode for normal use:

```bash
streamlit run app.py
```

```text
Streamlit UI -> application/services -> SQLite
```

FastAPI is not required. Local mode is selected whenever
`CHEMVAULT_API_URL` is not configured.

### API-client mode

API-client mode is intended for development and validation of the supported
Streamlit-to-FastAPI path.

Start FastAPI in one terminal:

```bash
python -m uvicorn api.main:app --reload
```

Start Streamlit in a second terminal.

Windows PowerShell:

```powershell
$env:CHEMVAULT_API_URL = "http://127.0.0.1:8000"
streamlit run app.py
```

macOS / Linux:

```bash
export CHEMVAULT_API_URL="http://127.0.0.1:8000"
streamlit run app.py
```

```text
Streamlit UI -> HTTP client -> FastAPI -> application/services -> SQLite
```

Interactive API documentation is available at
`http://127.0.0.1:8000/docs`. See [docs/api.md](docs/api.md) for the endpoint
contract and current limitations.

CHEMVAULT does not silently fall back to local execution when
`CHEMVAULT_API_URL` is configured and the backend is unavailable.

To return to local mode:

Windows PowerShell:

```powershell
Remove-Item Env:CHEMVAULT_API_URL -ErrorAction SilentlyContinue
streamlit run app.py
```

macOS / Linux:

```bash
unset CHEMVAULT_API_URL
streamlit run app.py
```

### Current API-client coverage

The HTTP path supports selected database inspection and export operations,
strict UniProt gene-target resolution, PubChem protein-search jobs, structure
consolidation, HARMONSMILE jobs, MODELABILITY INDEX jobs, and fingerprint
`.npz` export through the backend gateway.

Unsupported table mutations remain local workflows. The current backend runtime
is process-local and is not a distributed worker platform.

---

## Quick start

### From a CSV file

1. Start CHEMVAULT and open `http://localhost:8501`.
2. Enter a name in **SQL Database name**.
3. Upload a CSV file using **Upload CSV** in the sidebar.
4. Inspect the table using the **Columns** controls.
5. Use **Refine** to filter rows or columns.
6. Create a derived table when the selection should be preserved.
7. Use **Download CSV** to export the active selection.

### Supported organisms for gene-symbol resolution

CHEMVAULT provides a curated organism catalog for strict gene-symbol
resolution:

- `Homo sapiens` — Human — NCBI taxonomy ID `9606` (default);
- `Mus musculus` — Mouse — NCBI taxonomy ID `10090`;
- `Rattus norvegicus` — Rat — NCBI taxonomy ID `10116`;
- `Oryctolagus cuniculus` — Rabbit — NCBI taxonomy ID `9986`;
- `Mesocricetus auratus` — Golden hamster — NCBI taxonomy ID `10036`.

The catalog is backend-driven and deliberately ordered. Each gene-symbol query
remains singular: one gene symbol and one organism must resolve to exactly one
reviewed UniProt accession. Direct UniProt-accession input remains available
independently of this catalog.

### From PubChem to MODELABILITY INDEX

1. Start CHEMVAULT and enter a database name.
2. Select **Select Target**.
3. Choose one target-input mode:
   - enter a gene symbol and select one supported organism, then resolve it to
     one reviewed UniProt accession; `LEPR` with `Homo sapiens` (`9606`) resolves
     to `P48357`;
   - enter one UniProt accession directly; `P34971` is a practical smoke-test
     target.
4. Confirm the target and launch the PubChem search.
5. Follow the persisted PubChem job in the progress dialog. Completed searches
   populate `main`, `compound_assays`, and `compound_activities`.
6. Create or select a Structured activity subset for the activity type and
   population of interest.
7. Run **SMILES HARMONIZED** on its CID column.
8. Run **ACTIVITY LABELS** to preserve binary PubChem outcomes and consolidate
   repeated observations by `SMILES_Harmonized`.
9. Select the resulting consolidated table.
10. Choose a fingerprint in **MODELABILITY INDEX** and run the analysis.
11. Inspect the score, class concordance, structural context, provenance, and
    nearest-neighbor report.
12. Download the nearest-neighbor CSV, analysis JSON, or fingerprint `.npz`
    export.

> Databases are stored in `SQL/` and can be reopened in later sessions. Use a
> different database name when a completely fresh run is required.

---

## PubChem jobs and database persistence

PubChem protein searches are stored as persisted jobs in the selected SQLite
database. Long searches continue through the selected backend runtime while it
remains active, and the interface shows status, stage, progress, and messages.
Searches can be cancelled safely with **Cancel search**.

Existing target-based databases can rerun **Repair activity records** from
Database maintenance. Repair reconstructs protein-AID jobs from persisted
assay-compound relationships, retrieves current PubChem assay activity under
a shared request-level rate limit, and supports paginated retrieval for large
assays. Previously stored activity records are preserved, and each repair is
recorded in operation history with its outcome and diagnostics.

CHEMVAULT creates and maintains separate tables for molecular records, assay
metadata, and structured activity observations. Derived tables remain inside
the same database with provenance and operation history.

For newly completed target-based builds, CHEMVAULT persists the input mode and
canonical UniProt accession together with resolved gene, organism, protein,
and review-status metadata when available. The Database summary displays this
`Target identity` provenance. Legacy databases remain readable without
inferred or silently backfilled target metadata.

---

## Structured activity

Structured PubChem activity is stored in `compound_activities`, separately from
the main molecular table.

The interface supports filtering by fields such as:

- activity type;
- unit;
- PubChem outcome;
- AID;
- numeric activity range when a compatible activity type and unit are
  selected.

The on-screen preview may be bounded for responsiveness, while **Download CSV**
exports the complete filtered result.

Structured activity subsets provide the traceable inputs for the
**ACTIVITY LABELS** workflow.

---

## SMILES HARMONIZED

The **SMILES HARMONIZED** workflow is powered by
[HARMONSMILE](https://pypi.org/project/harmonsmile/). It accepts a valid PubChem
CID column and merges normalized molecular output back into the selected
table.

Important output fields include:

- `PubChem_CID` - canonical PubChem identifier;
- `PubChem_Acquisition_Status` - PubChem molecular acquisition outcome;
- `PubChem_Acquisition_Message` - optional diagnostic information for the
  acquisition stage;
- `SMILES_RDKit` - the RDKit-derived representation used during processing;
- `SMILES_Harmonized` - the final harmonized representation used by
  CHEMVAULT downstream workflows;
- `SMILES_Harmonization_Status` - `ok`, `ok_with_warnings`, `unsupported`, or
  `failed`;
- `SMILES_Harmonization_Message` - record-level diagnostic information;
- `InChI` and `InChIKey` when provided by the PubChem workflow.

PubChem acquisition and molecular harmonization are separate processing
stages. A successful acquisition can therefore coexist with a failed
harmonization result.

`SMILES_Harmonized`, not `SMILES_RDKit`, is the structural key used by
ACTIVITY LABELS, structure consolidation, and MODELABILITY INDEX.

---

## ACTIVITY LABELS and structure consolidation

The **ACTIVITY LABELS** workflow operates on eligible Structured activity
tables after SMILES harmonization.

It:

- preserves PubChem `Active` and `Inactive` outcomes;
- does not introduce pEC50, pKi, or other threshold-based labels;
- groups observations by `SMILES_Harmonized`;
- retains one structure-level record when the available observations agree;
- excludes conflicting structure-level outcomes traceably;
- preserves reference identifiers and contributing AIDs;
- leaves the source HARMONSMILE-enriched table unchanged.

The resulting table follows the
`activity_subset_<activity>_structure_consolidated` naming contract, with
numbered suffixes when necessary.

---

## MODELABILITY INDEX

The **MODELABILITY INDEX** evaluates whether binary activity labels are locally
consistent in molecular fingerprint space. For each eligible structure,
CHEMVAULT identifies its nearest neighbor by Tanimoto similarity and measures
same-class concordance for `Active` and `Inactive` structures. The final score
is the macro average of the two class concordances.

A value closer to 1 indicates stronger local agreement between molecular
similarity and the preserved binary labels. It is a dataset diagnostic, not a
guarantee of predictive performance.

### Structural context

CHEMVAULT complements the MODELABILITY INDEX with canonical atom- and
bond-aware Bemis-Murcko scaffold context. Cyclic molecules are partitioned by
scaffold, while acyclic molecules remain outside the Murcko partition and are
reported explicitly through coverage.

The result card presents this information in two groups. **Murcko population
context** reports Murcko population coverage, scaffold count, shared-scaffold
molecular coverage, and the adjusted scaffold effect. **Nearest-neighbor
scaffold interface** reports Murcko nearest-neighbor coverage, the same-scaffold
nearest-neighbor fraction, and same-scaffold and different-scaffold
concordances. These metrics provide interpretative context for the MODELABILITY
INDEX; they do not recalculate or alter the index itself.

### Fingerprint representations

Fingerprint generation is provided by
[MOLRAPTOR](https://pypi.org/project/molraptor/). CHEMVAULT supports the
following fingerprint representations:

- Morgan - default;
- Feature Morgan;
- Atom Pair;
- RDKit Topological;
- Topological Torsion;
- Layered;
- MACCS.

Analyses are scoped by database, source table, and fingerprint type. Compatible
completed results and persisted fingerprint artifacts can be restored and
reused without confusing different molecular representations.

CHEMVAULT records the resolved fingerprint profile, vector size, dependency
versions, analysis identity, Murcko scaffold definition, and related
provenance. It does not implement the fingerprint algorithms; encoding is
provided by MOLRAPTOR.

### Downloads

The result card provides:

- a complete nearest-neighbor CSV report containing the selected molecular
  comparisons together with the fingerprint type, analysis identity, and
  population identity;
- a JSON analysis report containing the MODELABILITY INDEX summary metrics,
  analysis provenance, nearest-neighbor tie and ambiguity diagnostics, and the
  existing Murcko population, structural metrics, and nearest-neighbor
  scaffold-interface context at full numeric precision;
- a self-contained fingerprint `.npz` export containing the fingerprint
  matrix, ordered harmonized SMILES, binary outcomes and labels, stable
  structure identifiers, and canonical provenance metadata.

The nearest-neighbor CSV and analysis JSON filenames include the fingerprint
type and the first eight characters of the CHEMVAULT analysis identity so that
outputs from different molecular representations remain distinguishable.

The `.npz` filename contract is:

```text
<database_id>_<activity_type>_fingerprints_<analysis8>.npz
```

---

## Table Manager and operation history

The **Table Manager** supports inspection of database tables, schema,
provenance, and operation history. Derived tables can be removed when they are
no longer needed. Database-maintenance controls are grouped separately from
normal curation and analysis actions.

---

## Running tests

Run the complete suite from the repository root:

```bash
python -m pytest tests -q
```

A local `.pytest_cache` permission warning can occur on some Windows systems.
It does not invalidate a successful test run.

---

## Contributing

See the [Contributing guide](CONTRIBUTING.md) for development setup,
validation, and pull-request guidance. Participation in CHEMVAULT is governed
by the [Code of Conduct](CODE_OF_CONDUCT.md).

---

## Related tools

CHEMVAULT integrates tools developed by the NanoBiostructures Research Group:

- [HARMONSMILE](https://pypi.org/project/harmonsmile/) - traceable SMILES
  harmonization;
- [MOLRAPTOR](https://pypi.org/project/molraptor/) - reproducible molecular
  fingerprint generation.

---

## Citation

To cite CHEMVAULT, use the citation metadata provided in
[CITATION.cff](CITATION.cff).

The recommended software citation is:

```text
Contreras-Torres, F. F., Murrieta, A. C., Castro-Flores, D., &
Saldívar-González, F. I. (2026).
CHEMVAULT — Traceable Target-Centered Bioactivity Dataset Construction and
Modelability Assessment (Version 0.14.7) [Computer software].
https://github.com/NanoBiostructuresRG-lab/chemvault
```

For reproducibility, cite the specific CHEMVAULT version used in the associated
methods, software, or data-availability section.

---

## License

CHEMVAULT is licensed under the
[GNU Lesser General Public License v3.0 or later](LICENSE).

See `LICENSE`, `COPYING`, and `COPYING.LESSER`.

SPDX identifier: `LGPL-3.0-or-later`.
