PlasRisk: A Multi-Dimension Data-Driven Weighted Risk Assessment Framework for Bacterial Plasmids
(A) Data-driven weight determination · (B) Command-line risk scoring · (C) Validation hierarchy · (D) Dimensionality analysis & lite mode
A. Data-driven weight determination
PIPdb database
792,964 PSCs
Train 634,359 / Test 158,605
10 risk dimensions
Each normalized to [0,1]
ARG · VF · MOB · SIZE · BM …
Training labels (4 outcomes)
● High-risk ARG carriage (3.1%)
● MDR–virulence fusion (1.8%)
● Conjugative mobility (5.1%)
● Biocide/metal resistance (34.0%)
Weight inference (3 methods)
① Random Forest MDG (non-linear)
② LASSO logistic regression (sparse)
③ Grid-search AUC optimization
Entropy & expert weights: comparators only
LORO-CV across 40 replicons
Consensus final weights
w = (w_RF + w_LASSO + w_grid) / 3
Σw = 1.000 · top-4 = 84.1%
Final weights:
S_ARG 0.245
S_BM 0.211
S_MOB 0.204
S_SIZE 0.181
S_VF 0.110
S_HOST 0.028
S_GROW 0.015
S_REP 0.003
S_HAB 0.002
S_GEO 0.002
Key finding: BMG–ARG co-occurrence
● S_BM = 2nd most important dimension (w = 0.211 vs expert 0.071)
● 60.3% of ARG+ plasmids co-carry BMGs (OR = 3.82)
● After removing 57 cross-annotated genes: OR = 3.09
● High-risk ARG plasmids: Cu/Ag enriched 2.9-fold
● 90.5% co-carry integrons/IS elements (physical linkage)
Composite risk score
S = Σ wᵢ · Sᵢ
S ∈ [0, 1] · Σwᵢ = 1.000
Consensus of RF-MDG + LASSO + Grid
--mode full (10-dim) | --mode lite (5-dim)
weights feed into tool
B. PlasRisk command-line tool (plasrisk *.fasta)
Input
Plasmid FASTA
*.fasta / *.fa / *.fna
single / batch / directory
Annotation (abricate)
CARD → ARGs (n, WHO, high-risk)
VFDB → VFs (exotoxin, T3SS/T4SS)
PlasmidFinder → replicon type
BacMet → biocide/metal genes
Sequence analysis
Length (log10 S_SIZE)
T4CP · relaxase · oriT · IS · integron (S_MOB)
Component scores (each 0–1)
S_ARGARG burden + WHO/high-risk
S_BMBiocide/metal resistance
S_MOBMobility + MGE plasticity
S_SIZECargo capacity (log10)
S_VFVirulence burden
S_HOSTHost range
S_GROWTemporal growth
S_REPReplicon prior
S_HABHabitat breadth
S_GEOGeographic spread
Lite mode: S_ARG + S_VF + S_MOB + S_SIZE + S_BM
Renormalized: 0.258 / 0.115 / 0.215 / 0.190 / 0.222
Mean AUC 0.920, equivalent to full model
Weighted sum
S = Σ wᵢ·Sᵢ
Risk grade
A ≥ 0.60 Very High
B ≥ 0.45 High
C ≥ 0.30 Moderate
■ D ≥ 0.15 ■ E < 0.15
Output
TSV / JSON
component scores
S_total, S_norm
grade (A–E)
replicon, mobility
high-risk genes
model_mode
(full / lite)
$ plasrisk *.fasta -o results/
$ plasrisk --mode lite *.fasta
# --no-abricate · --json · --mode {full,lite}
Replicon prior lookup: 66 typed replicons with pre-computed S_REP; unknown → neutral defaults
Missing metadata → conservative defaults; sensitivity: Cohen's κ ≥ 0.918, grade agreement ≥ 94.8%
multi-level validation
C. Validation hierarchy & key results
① Held-out test (20%, n=158,605)
● High-risk ARG: AUC = 0.958
● MDR–VF fusion: AUC = 0.964
● Conjugative: AUC = 0.856
● Biocide/metal: AUC = 0.903
● Mean AUC = 0.920
DeLong: consensus provides pragmatic balance
② LORO-CV & robustness
● LORO-CV (40 replicons): mean AUC = 0.962
● ±30% weight perturbation: ρ = 0.995
● Consensus aggregation sensitivity:
equal vs AUC-weighted: ρ = 1.000
● Logistic regression: S_BM independent
predictor (OR = 1.31, p = 0.033)
③ Calibration & DCA (natural prevalence)
● Stratified holdout: ROC = 0.968, PR = 0.413
● Grade A: 46.1% positive (vs 3.1% baseline)
● LR+ for Grade A = 27.2 (15-fold enrichment)
● DCA: positive net benefit at high-specificity
thresholds (HR: 0.54–0.75; BM: 0.12–0.75)
● Grades D/E (73.8%): 0 high-risk plasmids
④ Temporal & external validation
● Temporal split (pre/post-2020):
train mean AUC 0.924 → test 0.916
HR-ARG: 0.961 → 0.961 (p = 0.898, NS)
● External: 40 NCBI plasmids (balanced)
proof-of-concept only; independent
replication with ≥500 plasmids needed
Annotation–label circularity (transparent handling)
● S_ARG alone predicts high-risk ARG at AUC = 0.981 (label defined by ARG carriage)
● Composite model AUC = 0.958 (slightly lower due to weight dilution); value lies in multi-outcome stratification
● Removing S_ARG: AUC drops to 0.713; residual ~0.71 reflects genuine backbone signal (mobility, size, BM)
● Fixed-weight leave-one-dimension-out confirms cross-label prediction (e.g., no S_MOB → conj AUC = 0.685)
No overfitting (5 complementary diagnostics)
● Train–test AUC gap < 0.001; bootstrap optimism < 0.0003
● Learning curves converge at large N; adjusted AUC selects k = 5
● Fold stability: CV-AUC variation < 0.002 across 5 random folds
● All random processes use set.seed(42); conda env + sessionInfo provided
dimensionality optimization
D. Dimensionality analysis & PlasRisk-lite (all-subsets, 1,023 models, 5-fold CV)
All-subsets enumeration
2¹⁰ − 1 = 1,023 subsets
Weights renormalized per subset
AUC on 4 outcomes, 5-fold CV
Stepwise & LOO analysis
Forward: ARG→SIZE→MOB→BM→VF…
Fixed-weight LOO: cross-label AUCs
Removing BM causes largest single drop
Key results
● Plateau at k=5: mean AUC = 0.920
● 5-dim lite (ARG+VF+MOB+SIZE+BM)
● S_VF included: fusion AUC 0.943→0.963
DeLong tests (lite vs full)
● HR-ARG: Δ=0.000 (NS) · Conj: Δ=0.000
● Fusion: Δ=−0.001 (NS) · BM: Δ=+0.001
● All outcomes equivalent → 5-dim sufficient
Mean CV AUC vs. number of dimensions (plateau at k=5):
0.93
0.91
0.89
0.87
0.85
0.77
0.768
0.862
0.912
0.918
0.920
0.920
0.920
0.920
0.920
0.920
1
2
3
4
5
6
7
8
9
10
Plateau (k=5)
Number of dimensions
Full (10-dim) vs. Lite (5-dim) comparison
Metric
Full (10-dim)
Lite (5-dim)
Δ
High-risk ARG AUC
0.958
0.958
0.000
MDR–VF fusion AUC
0.964
0.963
−0.001
Conjugative AUC
0.856
0.856
0.000
Biocide/metal AUC
0.903
0.904
+0.001
Mean AUC (4 outcomes)
0.920
0.920
equivalent
Use case
One Health surveillance
Rapid screening
Full mode (10-dim): comprehensive One Health surveillance
● Includes host range, geography, habitat, growth, replicon priors for source attribution
● Pareto-optimal across all 4 outcomes; epidemiological context for policy & outbreak investigation
● CLI: plasrisk --mode full *.fasta
Lite mode (5-dim): rapid screening & resource-limited settings
● Requires only ARG + VFDB + mobility genes + length + BacMet (no metadata priors)
● Mean AUC = 0.920; S_VF raises fusion AUC to 0.963; 98.2% grade agreement with full
● CLI: plasrisk --mode lite *.fasta · Python: PlasRiskScorer(mode="lite")