DAY-EC activated.
DAY-EC activated.
=== status ===
{
  "command": "bin/day_run produce_alignstats produce_sentdont_snv_vcf produce_snv_concordances -p -j 5 -k",
  "completed_at": "2026-05-29T21:42:30Z",
  "exit_code": 1,
  "repo_path": "/fsx/analysis_results/ubuntu/ccv20260529r6_ont_snv_alignstats/daylily-omics-analysis",
  "session_name": "ccv20260529r6_ont_snv_alignstats",
  "started_at": "2026-05-29T21:22:54Z"
}
=== repo ===
/fsx/analysis_results/ubuntu/ccv20260529r6_ont_snv_alignstats/daylily-omics-analysis
=== latest snakemake logs ===
1780090947.0000000000 .snakemake/log/2026-05-29T212317.185118.snakemake.log
1780090952.0000000000 .snakemake/log/2026-05-29T214229.949942.snakemake.log
=== latest log path: .snakemake/log/2026-05-29T214229.949942.snakemake.log ===
=== latest log tail ===
Unlocking working directory.
=== rule logs with errors ===
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:11:[E::hts_open_format] Failed to open file "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/tp_vcf_gz_stripped.vcf.gz" : No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:12:[init_data] Error: cannot write to "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/tp_vcf_gz_stripped.vcf.gz": No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:13:tbx_index_build3 failed: results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/tp_vcf_gz_stripped.vcf.gz
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:15:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:28:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/tp_vcf_gz_stripped.vcf.gz'
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:30:During handling of the above exception, another exception occurred:
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:32:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:36:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/tp_vcf_gz_stripped.vcf.gz.X_tv_snp.vcf'
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:44:[E::hts_open_format] Failed to open file "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fp_vcf_gz_stripped.vcf.gz" : No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:45:[init_data] Error: cannot write to "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fp_vcf_gz_stripped.vcf.gz": No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:46:tbx_index_build3 failed: results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fp_vcf_gz_stripped.vcf.gz
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:48:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:61:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fp_vcf_gz_stripped.vcf.gz'
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:63:During handling of the above exception, another exception occurred:
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:65:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:69:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fp_vcf_gz_stripped.vcf.gz.X_tv_snp.vcf'
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:71:[E::hts_open_format] Failed to open file "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fn_vcf_gz_stripped.vcf.gz" : No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:72:[init_data] Error: cannot write to "results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fn_vcf_gz_stripped.vcf.gz": No such file or directory
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:73:tbx_index_build3 failed: results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fn_vcf_gz_stripped.vcf.gz
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:75:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:88:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fn_vcf_gz_stripped.vcf.gz'
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:90:During handling of the above exception, another exception occurred:
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:92:Traceback (most recent call last):
results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/logs/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.altair-v1.1.parse_vcfeval_summary.log:96:FileNotFoundError: [Errno 2] No such file or directory: 'results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/concordance/_altair-v1_1/fn_vcf_gz_stripped.vcf.gz.X_tv_snp.vcf'
logs/slurm/sent_snv_ont/sent_snv_ont.TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.1.err:61:        ulimit -n 65536 || echo "ulimit mod failed" >> results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/log/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.snv.log 2>&1;
logs/slurm/sent_snv_ont/sent_snv_ont.TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.1.err:100:            echo "ERROR: sentieon-cli dnascope-longread failed with exit code $cli_rc" >> results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/log/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.snv.log 2>&1;
logs/slurm/sent_snv_ont/sent_snv_ont.TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.1.err:111:            echo "ERROR: SNV VCF not produced by sentieon-cli" >> results/day/hg38_broad/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION/align/ont/na/snv/sentdont/log/TVBONT5X-HG003-5x-1-D0-PF-ONT-PROMETHION.ont.na.sentdont.snv.log 2>&1;
