Metadata-Version: 2.1
Name: cancerxpress
Version: 1.0.0
Summary: CancerXpress: pretrained RNA expression foundation model for batch correction and cancer downstream prediction
Author: CancerXpress Team
License: Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
           1. Definitions.
        
              "License" shall mean the terms and conditions for use, reproduction,
              and distribution as defined by Sections 1 through 9 of this document.
        
              "Licensor" shall mean the copyright owner or entity authorized by
              the copyright owner that is granting the License.
        
              "Legal Entity" shall mean the union of the acting entity and all
              other entities that control, are controlled by, or are under common
              control with that entity. For the purposes of this definition,
              "control" means (i) the power, direct or indirect, to cause the
              direction or management of such entity, whether by contract or
              otherwise, or (ii) ownership of fifty percent (50%) or more of the
              outstanding shares, or (iii) beneficial ownership of such entity.
        
              "You" (or "Your") shall mean an individual or Legal Entity
              exercising permissions granted by this License.
        
              "Source" form shall mean the preferred form for making modifications,
              including but not limited to software source code, documentation
              source, and configuration files.
        
              "Object" form shall mean any form resulting from mechanical
              transformation or translation of a Source form, including but
              not limited to compiled object code, generated documentation,
              and conversions to other media types.
        
              "Work" shall mean the work of authorship, whether in Source or
              Object form, made available under the License, as indicated by a
              copyright notice that is included in or attached to the work
              (an example is provided in the Appendix below).
        
              "Derivative Works" shall mean any work, whether in Source or Object
              form, that is based on (or derived from) the Work and for which the
              editorial revisions, annotations, elaborations, or other modifications
              represent, as a whole, an original work of authorship. For the purposes
              of this License, Derivative Works shall not include works that remain
              separable from, or merely link (or bind by name) to the interfaces of,
              the Work and Derivative Works thereof.
        
              "Contribution" shall mean any work of authorship, including
              the original version of the Work and any modifications or additions
              to that Work or Derivative Works thereof, that is intentionally
              submitted to Licensor for inclusion in the Work by the copyright owner
              or by an individual or Legal Entity authorized to submit on behalf of
              the copyright owner. For the purposes of this definition, "submitted"
              means any form of electronic, verbal, or written communication sent
              to the Licensor or its representatives, including but not limited to
              communication on electronic mailing lists, source code control systems,
              and issue tracking systems that are managed by, or on behalf of, the
              Licensor for the purpose of discussing and improving the Work, but
              excluding communication that is conspicuously marked or otherwise
              designated in writing by the copyright owner as "Not a Contribution."
        
              "Contributor" shall mean Licensor and any individual or Legal Entity
              on behalf of whom a Contribution has been received by Licensor and
              subsequently incorporated within the Work.
        
           2. Grant of Copyright License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              copyright license to reproduce, prepare Derivative Works of,
              publicly display, publicly perform, sublicense, and distribute the
              Work and such Derivative Works in Source or Object form.
        
           3. Grant of Patent License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              (except as stated in this section) patent license to make, have made,
              use, offer to sell, sell, import, and otherwise transfer the Work,
              where such license applies only to those patent claims licensable
              by such Contributor that are necessarily infringed by their
              Contribution(s) alone or by combination of their Contribution(s)
              with the Work to which such Contribution(s) was submitted. If You
              institute patent litigation against any entity (including a
              cross-claim or counterclaim in a lawsuit) alleging that the Work
              or a Contribution incorporated within the Work constitutes direct
              or contributory patent infringement, then any patent licenses
              granted to You under this License for that Work shall terminate
              as of the date such litigation is filed.
        
           4. Redistribution. You may reproduce and distribute copies of the
              Work or Derivative Works thereof in any medium, with or without
              modifications, and in Source or Object form, provided that You
              meet the following conditions:
        
              (a) You must give any other recipients of the Work or
                  Derivative Works a copy of this License; and
        
              (b) You must cause any modified files to carry prominent notices
                  stating that You changed the files; and
        
              (c) You must retain, in the Source form of any Derivative Works
                  that You distribute, all copyright, patent, trademark, and
                  attribution notices from the Source form of the Work,
                  excluding those notices that do not pertain to any part of
                  the Derivative Works; and
        
              (d) If the Work includes a "NOTICE" text file as part of its
                  distribution, then any Derivative Works that You distribute must
                  include a readable copy of the attribution notices contained
                  within such NOTICE file, excluding those notices that do not
                  pertain to any part of the Derivative Works, in at least one
                  of the following places: within a NOTICE text file distributed
                  as part of the Derivative Works; within the Source form or
                  documentation, if provided along with the Derivative Works; or,
                  within a display generated by the Derivative Works, if and
                  wherever such third-party notices normally appear. The contents
                  of the NOTICE file are for informational purposes only and
                  do not modify the License. You may add Your own attribution
                  notices within Derivative Works that You distribute, alongside
                  or as an addendum to the NOTICE text from the Work, provided
                  that such additional attribution notices cannot be construed
                  as modifying the License.
        
              You may add Your own copyright statement to Your modifications and
              may provide additional or different license terms and conditions
              for use, reproduction, or distribution of Your modifications, or
              for any such Derivative Works as a whole, provided Your use,
              reproduction, and distribution of the Work otherwise complies with
              the conditions stated in this License.
        
           5. Submission of Contributions. Unless You explicitly state otherwise,
              any Contribution intentionally submitted for inclusion in the Work
              by You to the Licensor shall be under the terms and conditions of
              this License, without any additional terms or conditions.
              Notwithstanding the above, nothing herein shall supersede or modify
              the terms of any separate license agreement you may have executed
              with Licensor regarding such Contributions.
        
           6. Trademarks. This License does not grant permission to use the trade
              names, trademarks, service marks, or product names of the Licensor,
              except as required for reasonable and customary use in describing the
              origin of the Work and reproducing the content of the NOTICE file.
        
           7. Disclaimer of Warranty. Unless required by applicable law or
              agreed to in writing, Licensor provides the Work (and each
              Contributor provides its Contributions) on an "AS IS" BASIS,
              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
              implied, including, without limitation, any warranties or conditions
              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
              PARTICULAR PURPOSE. You are solely responsible for determining the
              appropriateness of using or redistributing the Work and assume any
              risks associated with Your exercise of permissions under this License.
        
           8. Limitation of Liability. In no event and under no legal theory,
              whether in tort (including negligence), contract, or otherwise,
              unless required by applicable law (such as deliberate and grossly
              negligent acts) or agreed to in writing, shall any Contributor be
              liable to You for damages, including any direct, indirect, special,
              incidental, or consequential damages of any character arising as a
              result of this License or out of the use or inability to use the
              Work (including but not limited to damages for loss of goodwill,
              work stoppage, computer failure or malfunction, or any and all
              other commercial damages or losses), even if such Contributor
              has been advised of the possibility of such damages.
        
           9. Accepting Warranty or Additional Liability. While redistributing
              the Work or Derivative Works thereof, You may choose to offer,
              and charge a fee for, acceptance of support, warranty, indemnity,
              or other liability obligations and/or rights consistent with this
              License. However, in accepting such obligations, You may act only
              on Your own behalf and on Your sole responsibility, not on behalf
              of any other Contributor, and only if You agree to indemnify,
              defend, and hold each Contributor harmless for any liability
              incurred by, or claims asserted against, such Contributor by reason
              of your accepting any such warranty or additional liability.
        
           END OF TERMS AND CONDITIONS
        
           APPENDIX: How to apply the Apache License to your work.
        
              To apply the Apache License to your work, attach the following
              boilerplate notice, with the fields enclosed by brackets "[]"
              replaced with your own identifying information. (Don't include
              the brackets!)  The text should be enclosed in the appropriate
              comment syntax for the file format. We also recommend that a
              file or class name and description of purpose be included on the
              same "printed page" as the copyright notice for easier
              identification within third-party archives.
        
           Copyright [yyyy] [name of copyright owner]
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
        
Project-URL: Repository, https://github.com/Qiankun-1024/CancerXpress
Project-URL: Documentation, https://github.com/Qiankun-1024/CancerXpress#readme
Project-URL: Issues, https://github.com/Qiankun-1024/CancerXpress/issues
Classifier: Programming Language :: Python :: 3.8
Classifier: Operating System :: POSIX :: Linux
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: <3.9,>=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tensorflow-gpu==2.4.0
Requires-Dist: tensorflow-addons==0.13.0
Requires-Dist: numpy==1.19.5
Requires-Dist: pandas==1.2.0
Requires-Dist: scipy==1.5.3
Requires-Dist: scikit-learn==1.3.0
Requires-Dist: joblib==1.3.1
Requires-Dist: requests==2.32.4
Requires-Dist: protobuf==3.20.3

# CancerXpress

CancerXpress is a pan-cancer transcriptomic representation learning framework for bulk RNA-seq data. It learns de-batched expression profiles and transferable biological representations that can be used for downstream prediction, functional state analysis, and interpretable single-sample characterization.

It is designed for use cases where transcriptomic models need to generalize across cohorts, sequencing workflows, and cancer types rather than only perform well within a single dataset.

## Key Features

- batch effect correction and corrected expression reconstruction
- latent embedding extraction for downstream analysis
- pan-cancer representation learning across cohorts and cancer types
- Module Eigenpathway prediction
- survival risk prediction with external cohort evaluation
- functional state characterization through low-dimensional transcriptomic axes
- sample-level attribution through integrated gradients
- hierarchical gene-to-axis-to-risk attribution with exact ME Shapley,
  matched baselines, and completeness diagnostics
- automatic gene ID conversion between `Ensembl` IDs and `HGNC symbols`
- training and fine-tuning scripts under `training/`

This release provides representation/batch correction, functional-axis and risk
prediction, attribution, reference search and target-cohort adaptation. It does
not provide cancer-type or primary-site prediction, plotting functions, or
automatic reference-data installation. Cancer type remains a user-supplied
clinical/biological label. Outputs are for research, not clinical diagnosis.

## Quick Start

Install from a local checkout (Python 3.8; see [Installation](#installation)
for direct GitHub installation):

```bash
cd CancerXpress
pip install -r requirements.txt
pip install -e .
```

Run the full inference pipeline on the example data:

```bash
cancerxpress \
  --input examples/scanb_demo_3samples.tsv \
  --clinical-file examples/scanb_demo_3samples_label.tsv \
  --outdir output/demo \
  --task all \
  --gene-id-type ensembl
```

Run survival risk prediction only:

```bash
cancerxpress \
  --input examples/scanb_demo_3samples.tsv \
  --clinical-file examples/scanb_demo_3samples_label.tsv \
  --outdir output/risk_demo \
  --task risk \
  --gene-id-type ensembl
```

Use the Python API:

```python
import pandas as pd
import cancerxpress as cx

expr = pd.read_csv('examples/scanb_demo_3samples.tsv', sep='\t', index_col=0)
clinical = pd.read_csv('examples/scanb_demo_3samples_label.tsv', sep='\t', index_col=0)

model = cx.CancerXpress()
latent, corrected = model.batch_correct(expr, gene_id_type='ensembl')
module_eigenpathway_predictions = model.predict_me(expr, gene_id_type='ensembl')
risk_scores = model.predict_survival_risk(expr, clinical=clinical, gene_id_type='ensembl')
```

Run the standalone survival risk example:

```bash
python examples/risk_prediction_demo.py
```

Run the standalone survival risk attribution example:

```bash
python examples/risk_attribution_demo.py
```

## Project Layout

- `cancerxpress/`: installable Python package
- `resources/`: required assets, encoders, mappings, and model manifests/checkpoints
- `training/`: key training and fine-tuning scripts
- `examples/`: small example input files

## Packaged Models

- `resources/models/pretrained/batch_correction/`
  Model parameters for batch correction, latent embedding extraction, and corrected expression reconstruction.
- `resources/models/finetuned/me/`
  Model parameters for Module Eigenpathway prediction.
- `resources/models/finetuned/survival_risk/`
  Model parameters for survival risk prediction.

Model parameters and reference data are installed manually and are not included
in this package. Keep the supplied bundle layout unchanged; older bundles may
contain unused classification checkpoints, which this API does not load.

## Installation

The current package requires **Python 3.8** and pins TensorFlow GPU 2.4.0
and its companion dependencies. Use a dedicated environment; installation
with newer Python versions is not supported by the current package metadata.

### Install Directly from GitHub

With Git installed, run in your Python 3.8 environment:

```bash
python -m pip install "git+https://github.com/Qiankun-1024/CancerXpress.git@main"
```

For a reproducible installation, replace `main` with a specific commit hash
or release tag. This installs the Python package and its declared dependencies;
it does **not** download the large model checkpoints or create a local checkout
of the example files.

Prepare the local model files as described in [Model Resources and Paths](#model-resources-and-paths)
before running predictions.
The package is tested in an isolated Python 3.8 environment. Use
`requirements-release-lock.txt` as pip constraints for the tested dependency
versions. The official Zenodo download URL is pending publication.

### Install from a Local Checkout

Use this option to access the examples or edit the source:

```bash
git clone https://github.com/Qiankun-1024/CancerXpress.git
cd CancerXpress
python -m pip install -c requirements-release-lock.txt -r requirements.txt
python -m pip install -c requirements-release-lock.txt -e .
```

### Model Resources and Paths

The wheel and source distribution contain **code only**, not model weights or
patient data. The resource bundle includes inference checkpoints, gene mapping,
normalization assets, label encoders and clinical preprocessing metadata.

```python
import cancerxpress as cx

# Manually extract the model bundle to ~/.cache/cancerxpress/resources first.
# Construction and prediction never download missing files automatically.
model = cx.CancerXpress()

# Alternatively, use your own extracted bundle:
model = cx.CancerXpress(resource_dir="/path/to/resources")
# Override only weights if needed (assets still come from resource_dir):
model = cx.CancerXpress(
    resource_dir="/path/to/resources", model_dir="/path/to/other/models"
)
cx.verify_resources("/path/to/resources")
```

Set `CANCERXPRESS_HOME` to change the default resource root. Paths do not depend
on the working directory. CLI equivalents are `--resource-dir` and `--model-dir`.
For a local checkout using its existing resources, pass `resource_dir="resources"`
explicitly. Do not mix weights and preprocessing assets from different releases.

The downloader requires HTTPS and the expected SHA-256, verifies every file,
rejects unsafe archive paths, and never overwrites an existing destination.
Only install bundles from a trusted publisher: label encoders use Python pickle.
See [Manually Install Reference Data](#manually-install-reference-data) for the
resource and reference-directory layout. No published download URL is assumed.

## Python API

CancerXpress provides a single high-level entry point for preprocessing, prediction, and attribution:

```python
import pandas as pd
import cancerxpress as cx

expr = pd.read_csv('examples/scanb_demo_3samples.tsv', sep='\t', index_col=0)
clinical = pd.read_csv('examples/scanb_demo_3samples_label.tsv', sep='\t', index_col=0)

model = cx.CancerXpress()
latent, corrected = model.batch_correct(expr, gene_id_type='ensembl')
module_eigenpathway_predictions = model.predict_me(expr, gene_id_type='ensembl')
risk_scores = model.predict_survival_risk(
    expr,
    clinical=clinical,
    gene_id_type='ensembl',
)

module_eigenpathway_attribution = model.attribute_me(expr.iloc[[0]], me_name='MEblue', gene_id_type='ensembl')
survival_risk_attribution = model.attribute_survival_risk(
    expr.iloc[[0]],
    clinical=clinical.iloc[[0]],
    gene_id_type='ensembl',
)
```

Main API methods:

- `batch_correct(...)`
- `predict_me(...)`
- `predict_survival_risk(...)`
- `attribute_me(...)`
- `attribute_survival_risk(...)`
- `attribute_axis_risk(...)`

Attribution covers functional axes and survival risk, not classification.

## Survival Risk Prediction

The packaged survival risk model is the run1 ME-and-clinical Cox model. CancerXpress first predicts the eight Module Eigenpathway scores from expression and then combines them with the clinical covariates.
It requires:

- an expression matrix
- a clinical table containing `age`, `sex`, `race`, `tumor_stage`, and `cancer_type`

### Required Input Format

Expression matrix:

- rows: samples
- columns: genes
- values: TPM-like expression matrix accepted by CancerXpress

Clinical file:

- same sample order as the expression matrix, or matching sample IDs in the first column
- must contain `age`, `sex`, `race`, `tumor_stage`, and `cancer_type`

Example clinical file:

```tsv
sample_id	age	sex	race	tumor_stage	cancer_type
sample_1	62	female	white	Stage II	BRCA
sample_2	55	male	asian	Stage III	LUAD
```

### Python Example

```python
import pandas as pd
import cancerxpress as cx

expr = pd.read_csv('examples/scanb_demo_3samples.tsv', sep='\t', index_col=0)
clinical = pd.read_csv('examples/scanb_demo_3samples_label.tsv', sep='\t', index_col=0)

model = cx.CancerXpress()
risk_scores = model.predict_survival_risk(
    expr_tpm=expr,
    clinical=clinical,
    gene_id_type='ensembl',
)
print(risk_scores)
```

### Survival Risk Attribution Example

```python
import pandas as pd
import cancerxpress as cx

expr = pd.read_csv('examples/scanb_demo_3samples.tsv', sep='\t', index_col=0)
clinical = pd.read_csv('examples/scanb_demo_3samples_label.tsv', sep='\t', index_col=0)

sample_id = expr.index[0]

model = cx.CancerXpress()
attribution = model.attribute_survival_risk(
    expr_tpm=expr.loc[[sample_id]],
    clinical=clinical.loc[[sample_id]],
    gene_id_type='ensembl',
)
```

### Reference Queries and Target-Cohort Adaptation

#### Manually Install Reference Data

The supported setup is currently **local files supplied separately**. No data
hosting service, DOI, account or automatic download is required. Installing the
Python package does not install model parameters or the reference database.
Missing files produce an error rather than trigger a download.

Place the files in this layout (or choose any absolute directory):

```text
~/.cache/cancerxpress/
  resources/                     # Model bundle, required for inference
    assets/                      # Keep the supplied bundle layout unchanged
    models/
    encoders/
    gene_annotation/
    mappings/
    resource_manifest.json
  reference/                     # Indexed data release, for sample retrieval
    reference_index.json.gz
    expression_samples_001.zip
    expression_samples_002.zip
    ...                          # Every ZIP named in reference_index.json.gz
    reference_atlas/              # Optional fast-loading query/adaptation atlas
      atlas.json
      samples.tsv
      features.npz
```

Do not unzip or rename the indexed ZIP files. They contain raw TPM, corrected
expression, latent, ME and sample metadata; these five components need not be
copied again as separate tables. ZIP counts are defined by the release index,
not hard-coded in the package.

| Use case | Required reference files |
| --- | --- |
| Predict or attribute your own TPM | None; only the matching model/resource bundle |
| Query or adapt against a supplied atlas | `atlas.json`, `samples.tsv`, `features.npz` |
| Retrieve stored sample expression/features | `reference_index.json.gz` and all indexed ZIP shards |

`atlas.json` records the format, feature column names and model/preprocessing
signature. `features.npz` contains numeric `latent` and `me` arrays in exactly the
row order of `samples.tsv`. The latter is a sample-indexed TSV with `sample_id`
and `cancer_type`; additional biological labels are optional. Preserve missing
reference labels. Do not combine feature files and models from different releases.

#### Read Specific Reference Samples

The indexed reference release uses uniform sample shards, not cancer-specific
archives. Each sample's TPM, corrected expression and features are independently
compressed inside an uncompressed ZIP container. A small index maps sample IDs
to exact byte ranges and SHA-256 checksums. This is different from the historical
whole-file `.tar.gz` release; those files cannot use this interface.

```python
from cancerxpress import SampleDatabase

# Default: ~/.cache/cancerxpress/reference, or CANCERXPRESS_REFERENCE_DIR.
db = SampleDatabase.from_directory()
# Alternatively: db = SampleDatabase.from_directory("/data/cancerxpress/reference")
ids = db.sample_ids[:2]  # Replace with the desired IDs from this release.
data = db.fetch(ids, components=["raw_tpm", "me", "metadata"])
# data["raw_tpm"]: samples x genes; data["me"]: samples x functional axes.
```

Supported components: `raw_tpm`, `corrected_expression`, `latent`, `me`, `metadata`.
Raw TPM and corrected expression preserve their original, potentially different
gene columns. Missing metadata labels are not inferred. Latent/ME use the frozen
release model recorded in the index. Opening the directory checks shard presence
and sizes; reading a component verifies its SHA-256. Local reads do not use the
network or create duplicate component caches. Only requested components are read.
`CANCERXPRESS_REFERENCE_DIR` controls reference data only; `CANCERXPRESS_HOME`
controls model resources. Neither default depends on the working directory.
Automatic data download can be added later without changing the local format.
The existing explicit-URL transport is optional and is not needed for this workflow.

A reference atlas stores sample-indexed latent means, eight ME values and
biological labels generated with one frozen model. It does not ship inside the
Python wheel. Use a supplied TCGA atlas or build one from reference TPM:

```python
atlas = cx.ReferenceAtlas.load("/data/cancerxpress/reference/reference_atlas")
# If only the indexed release is available, read its features (not expression):
# atlas = db.to_atlas()  # All samples; can take time on a large reference.
# atlas.save("/data/cancerxpress/reference/reference_atlas")  # New directory only.
# Or build your own atlas from reference TPM with model.build_reference(...).

hits = model.query(
    new_tpm, atlas, metadata=new_metadata,
    space="latent", k=10, metric="cosine", same_cancer=True,
    gene_id_type="ensembl",
)
# Use space="me" for functional-axis similarity; same_cancer=False for cross-cancer search.
```

Expression tables are samples x genes in TPM; metadata is indexed by the same
sample IDs and contains `cancer_type`. Missing reference cancer labels remain
missing (never inferred or imputed) and are allowed in unrestricted queries.
`same_cancer=True` requires a nonmissing query label; unlabelled reference rows
are excluded. Nonmissing query cancer labels must occur in the atlas.
Extra labels such as `subtype` or `grade` are optional.
Feature spaces and checkpoint/preprocessing hashes must match. Query distances
default to reference-only z-scored features; cosine similarity is 1-distance.
Euclidean mode reports distance (similarity is NaN). These are not confidence
scores or prognostic predictions. Self-exclusion uses exact sample ID.

Fit a shared target-only residual adapter while keeping the encoder/ME head frozen:

```python
fit = model.adapt(
    new_tpm, new_metadata, atlas,
    biological_labels=["subtype"],  # Omit for cancer-only alignment, including a single cancer.
    epochs=200, patience=25, gene_id_type="ensembl",
)
fit.save("cohort_adapter")
print(fit.metrics)  # Before/after held-out MMD and displacement, not clinical validation.

adapter = cx.LatentAdapter.load_adapter("cohort_adapter")
latent, me = model.encode_features(
    new_tpm, adapter=adapter, metadata=new_metadata, gene_id_type="ensembl"
)
hits = model.query(
    new_tpm, atlas, metadata=new_metadata, adapter=adapter,
    space="me", same_cancer=True, gene_id_type="ensembl"
)
```

Cancer type is ALWAYS part of the alignment strata. Additional labels must exist
in both cohorts; unseen strata are rejected. No survival or NMF program labels
are used automatically. Use genuine biology labels, not outcome-derived groups.
Reference rows missing any requested biological label are excluded from fitting;
target rows missing these labels are rejected rather than silently reassigned.
Each stratum needs at least five independent patients in each cohort to create
60/20/20-style training/validation/test splits; that minimum is a software check,
not evidence that a tiny cohort is statistically adequate.
Provide `patient_id` to keep repeated assays in one partition. Otherwise IDs
are assumed to identify independent patients.

The objective is equal-weight per-stratum RBF MMD + 0.05 latent displacement
+ 0.02 pairwise-distance distortion. Only reference TRAINING samples determine
normalization. Validation-only early stopping selects weights. Validation/test
MMD uses at most the first 256 samples per stratum, deterministically; metrics
record evaluated counts. Inspect saved split tables and biological preservation;
smaller MMD is not proof of biological recovery. Adapter use is explicit and
does not change the reference atlas or ordinary prediction/attribution calls.
Current adapter support covers `encode_features` and `query`; risk/IG APIs do
not automatically apply it. Do not mix adapted ME with old gene attribution.
Training/validation patients are not untouched external validation.

Historical TCGA OOF attribution tables and fixed-release-model query features
are distinct outputs. Use the companion TCGA handoff README for definitions,
QC, patient/gene ordering and the historical baseline differences.

## CLI Example

```bash
cancerxpress \
  --input examples/scanb_demo_3samples.tsv \
  --clinical-file examples/scanb_demo_3samples_label.tsv \
  --outdir output/risk_demo \
  --task risk \
  --gene-id-type ensembl
```

### Output

The risk prediction output is a table with one row per sample and one column:

- `risk_score`: relative Cox-model risk score

Interpretation:

- a higher `risk_score` means a higher predicted relative risk
- the score is mainly useful for ranking samples within a cohort or cancer setting
- it is not a direct survival probability

For attribution output:

- `survival_risk_prediction.tsv`: predicted risk score for the input sample
- `survival_risk_attribution.tsv`: gene-level integrated gradients attribution for survival risk

### Axis-Specific Risk Attribution

Pass the patient's TPM, clinical information, and reference TPM samples.
The package handles internal image conversion and computes exact ME Shapley,
gene-to-axis IG, and the selected axis's gene-level risk allocation.

```python
import pandas as pd
import cancerxpress as cx

expr = pd.read_csv('patient_tpm.tsv', sep='\t', index_col=0)
clinical = pd.read_csv('clinical.tsv', sep='\t', index_col=0)
reference = pd.read_csv('training_reference_tpm.tsv', sep='\t', index_col=0)

result = cx.CancerXpress().attribute_axis_risk(
    expr.iloc[[0]], clinical.loc[expr.index[:1]],
    me_name='MEred', reference_tpm=reference, gene_id_type='ensembl',
)
result.axis_shapley.to_csv('axis_shapley.tsv', sep='\t')
result.gene_risk_contribution.to_csv('gene_axis_risk.tsv', sep='\t')
print(result.diagnostics)
```

- Both TPM matrices have samples as rows and genes as columns. Use a fixed
  training reference from the corresponding cancer/model/fold, not the test
  patient itself or an external cohort recentered separately. Reference patients
  do not need clinical information; the target patient's clinical inputs stay fixed.
- Internally, reference samples are individually preprocessed and their model-input
  images averaged. The model prediction of this average supplies the ME baseline.
  No image or ME-reference file is required from the user. This baseline is NOT
  the mean of individual ME predictions and may differ from earlier research
  results using an optimized training-mean-ME baseline.
- Outputs: `axis_shapley` (eight axes), `gene_to_axis_ig`,
  `gene_risk_contribution`, and `diagnostics` (completeness errors and validity).
  Allocation is `axis_shapley * gene_IG / sum(gene_IG)`, over all mapped genes.
  Values are signed log-risk allocations, not percentages or causal effects.
  Invalid allocations become NaN.
- This differs from direct gene-to-risk IG (`attribute_survival_risk`).
  Advanced primitives remain available: `exact_axis_shapley`,
  `gene_axis_integrated_gradients`, `allocate_axis_risk`, and
  `optimize_axis_baseline`.

## CLI

### Run the Full Inference Pipeline

```bash
cancerxpress \
  --input examples/scanb_demo_3samples.tsv \
  --clinical-file examples/scanb_demo_3samples_label.tsv \
  --outdir output/demo \
  --task all \
  --gene-id-type ensembl
```

Expected outputs:

- `latent.tsv`
- `corrected_expression.tsv`
- `me_predictions.tsv`
- `survival_risk.tsv`

### Run Attribution from CLI

```bash
cancerxpress \
  --input examples/scanb_demo_3samples.tsv \
  --outdir output/attr_demo \
  --task attribution \
  --attribution-task me \
  --target-name MEblue \
  --sample-id TCGA-A2-A0T2-01A \
  --gene-id-type ensembl
```

Expected outputs:

- `attribution.tsv`
- `attribution_prediction.tsv`

### Input Expression with HGNC Symbols

If the input matrix uses gene symbols instead of Ensembl IDs:

```bash
cancerxpress \
  --input your_symbol_expression.tsv \
  --clinical-file your_clinical_data.tsv \
  --outdir output/run_symbol \
  --task all \
  --gene-id-type symbol
```

## Training and Fine-Tuning

The GitHub checkout contains research training scripts under `training/`.
They are not installed by the wheel or source distribution and are not a
unified, supported pretraining API. The supported package adaptation entry point
is `model.adapt(...)`, described above. Repository-only scripts include:

- `training/ME_regression.py`: regression training for eight Module Eigenpathway targets
- `training/improved_risk_prediction.py`: survival risk model training

### Example: Module Eigenpathway Training

```bash
python training/ME_regression.py \
  --data /path/to/expression_matrix.tsv \
  --label /path/to/me_targets.tsv \
  --ckpt-dir training_checkpoints/me_regressor
```

### Example: Survival Risk Training

```bash
python training/improved_risk_prediction.py \
  --data /path/to/expression_matrix.tsv \
  --label /path/to/survival_labels.tsv \
  --test-batches external_batch_1 external_batch_2 \
  --holdout-cancer-types optional_cancer_type
```

## What Was Not Migrated

The following content was intentionally not copied into CancerXpress:

- most prediction outputs under `output/`
- historical training logs
- historical checkpoints
- very large internal training matrices
- attribution scripts and attribution intermediate results

This keeps the project much cleaner for packaging, publication, and long-term maintenance.

## Notes

- The packaged run1 survival model requires expression plus `age`, `sex`, `race`, `tumor_stage`, and `cancer_type` for every sample.
- Training scripts do not automatically download datasets. You need to provide your own expression matrices and labels.
