DAY-EC activated.
DAY-EC activated.
=== PATTERN Error in rule ===
3152-    reason: Missing output files: results/day/hg38/other_reports/peddy_sample_qc_mqc.tsv; Input files updated by another job: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.done
3153-    resources: mem_mb=3000, mem_mib=2862, disk_mb=1000, disk_mib=954, tmpdir=/dev/shm, threads=1, time=5440, partition=i192,i128,i192mem, vcpu=1, distribution=block, exclusive=, constraint=, exclude=, include=
3154-
3155-[Sat May 30 10:40:32 2026]
3156:Error in rule read_haps_contam_identity:
3157-    jobid: 84
3158-    input: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam.bai, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz.tbi
3159-    output: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
3160-    log: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log (check log file(s) for error details)
3161-    conda-env: /fsx/resources/environments/conda/ubuntu/ip-10-0-0-88/efba687ed98dc6d0ecc827ba5c267513_
3162-    shell:
3163-
3164-        set -euo pipefail
3165-        test ok = ok
3166-        mkdir -p "$(dirname results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt)" "$(dirname results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log)"
3167-        set +o pipefail
3168-        if gzip -cd results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz | grep -m 1 -q -v '^#'; then
3169-            has_variants=true
3170-        else
3171-            has_variants=false
3172-        fi
3173-        set -o pipefail
3174-        if [[ "$has_variants" == "true" ]]; then
3175-            command -v /fsx/references/runtime_assets/tool_specific_resources/read_haps/read_haps > /dev/null
3176-            test -s /fsx/references/runtime_assets/tool_specific_resources/read_haps/high_quality_markers_deCODE_2015.txt.gz
3177-            /fsx/references/runtime_assets/tool_specific_resources/read_haps/read_haps  -fa /fsx/references/genomic_data/organism_references/H_sapiens/hg38/fasta_fai_minalt/GRCh38_no_alt_analysis_set.fasta results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam /fsx/references/runtime_assets/tool_specific_resources/read_haps/high_quality_markers_deCODE_2015.txt.gz results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz > results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt 2> results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log
3178-        else
3179-            printf 'SNP_PAIRS ERROR_PAIRS DOUBLE_ERROR_PAIR_COUNT DOUBLE_ERROR_FRACTION REL_ERROR_FRACTION NONSENSE_FRACTION PASS_FAIL REASON
3180-0 0 0 0 0 0 NO_DATA NO_VARIANTS
3181-' > results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
--
4158-[Sat May 30 10:40:37 2026]
4159-Finished job 125.
4160-91 of 161 steps (57%) done
4161-[Sat May 30 10:40:44 2026]
4162:Error in rule read_haps_contam_identity:
4163-    jobid: 84
4164-    input: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam.bai, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz, results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz.tbi
4165-    output: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
4166-    log: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log (check log file(s) for error details)
4167-    conda-env: /fsx/resources/environments/conda/ubuntu/ip-10-0-0-88/efba687ed98dc6d0ecc827ba5c267513_
4168-    shell:
4169-
4170-        set -euo pipefail
4171-        test ok = ok
4172-        mkdir -p "$(dirname results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt)" "$(dirname results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log)"
4173-        set +o pipefail
4174-        if gzip -cd results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz | grep -m 1 -q -v '^#'; then
4175-            has_variants=true
4176-        else
4177-            has_variants=false
4178-        fi
4179-        set -o pipefail
4180-        if [[ "$has_variants" == "true" ]]; then
4181-            command -v /fsx/references/runtime_assets/tool_specific_resources/read_haps/read_haps > /dev/null
4182-            test -s /fsx/references/runtime_assets/tool_specific_resources/read_haps/high_quality_markers_deCODE_2015.txt.gz
4183-            /fsx/references/runtime_assets/tool_specific_resources/read_haps/read_haps  -fa /fsx/references/genomic_data/organism_references/H_sapiens/hg38/fasta_fai_minalt/GRCh38_no_alt_analysis_set.fasta results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/compat_bam/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.legacy_compat.bam /fsx/references/runtime_assets/tool_specific_resources/read_haps/high_quality_markers_deCODE_2015.txt.gz results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.snv.sort.vcf.gz > results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt 2> results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/logs/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.log
4184-        else
4185-            printf 'SNP_PAIRS ERROR_PAIRS DOUBLE_ERROR_PAIR_COUNT DOUBLE_ERROR_FRACTION REL_ERROR_FRACTION NONSENSE_FRACTION PASS_FAIL REASON
4186-0 0 0 0 0 0 NO_DATA NO_VARIANTS
4187-' > results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
=== PATTERN RuleException ===
=== PATTERN Job failed ===
=== PATTERN exited with ===
1885-                printf 'WARNING: peddy reported no usable heterozygous variants; writing explicit low-data QC outputs
1886-' | tee -a "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/log/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.log" >&2
1887-                python bin/util/write_peddy_low_data_outputs.py                     --prefix "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy."                     --ped "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.ped"                     --sample-id "JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ"                     --reason "peddy_no_usable_heterozygous_variants"                     >> "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/log/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.log" 2>&1
1888-            else
1889:                printf 'ERROR: peddy exited with status %s
1890-' "$peddy_status" | tee -a "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/log/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.log" >&2
1891-                exit "$peddy_status"
1892-            fi
1893-        }
1894-
1895-        for expected_output in             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.ped"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.ped_check.csv"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.sex_check.csv"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.het_check.csv"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.background_pca.json"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.html"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.vs.html"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.ped_check.png"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.het_check.png"             "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.sex_check.png"
1896-        do
1897-            if [[ ! -s "$expected_output" ]]; then
1898-                printf 'ERROR: peddy completed but expected output is missing or empty: %s
1899-' "$expected_output" | tee -a "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/log/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.log" >&2
1900-                exit 1
1901-            fi
1902-        done
1903-
1904-        printf 'reallydone
1905-' > "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.done"
1906-        printf 'reallydone
1907-' > "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy."
1908-        printf 'DONE
1909-' >> "results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/peddy/log/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.peddy.log"
1910-
1911-Submitted job 76 with external jobid '3345'.
1912-
1913-[Sat May 30 10:40:10 2026]
1914-rule haplocheck_vcf_contam_identity:
--
3185-        test -s results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
3186-        grep -q 'PASS_FAIL' results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
3187-        grep -q 'REASON' results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
3188-
3189:        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
3190-    cluster_jobid: 3355
3191-
3192-Error executing rule read_haps_contam_identity on cluster (jobid: 84, external: 3355, jobscript: /fsx/analysis_results/ubuntu/ccv20260529r40_illumina_hg002_kitchensink_multiqc/daylily-omics-analysis/.snakemake/tmp.yun4xh_6/snakejob.read_haps_contam_identity.84.sh). For error details see the cluster log and the log files of the involved rule(s).
3193-[Sat May 30 10:40:32 2026]
3194-Finished job 122.
3195-61 of 161 steps (38%) done
3196-Select jobs to execute...
3197-
3198-[Sat May 30 10:40:32 2026]
3199-localrule bcftools_variant_stats_gather:
3200-    input: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/bcfstats/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.bcfstats.tsv
3201-    output: results/day/hg38/other_reports/bcftools_variant_stats_mqc.tsv
3202-    jobid: 121
3203-    reason: Missing output files: results/day/hg38/other_reports/bcftools_variant_stats_mqc.tsv; Input files updated by another job: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/bcfstats/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.bcfstats.tsv
3204-    resources: mem_mb=3000, mem_mib=2862, disk_mb=1000, disk_mib=954, tmpdir=/dev/shm, threads=1, time=5440, partition=i192,i128,i192mem, vcpu=1, distribution=block, exclusive=, constraint=, exclude=, include=
3205-
3206-[Sat May 30 10:40:32 2026]
3207-Finished job 124.
3208-62 of 161 steps (39%) done
3209-Trying to restart job 84.
3210-Select jobs to execute...
3211-
3212-[Sat May 30 10:40:32 2026]
3213-localrule rtg_vcfstats_gather:
3214-    input: results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/vcf_stats/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.rtg.vcfstats.txt
--
4191-        test -s results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
4192-        grep -q 'PASS_FAIL' results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
4193-        grep -q 'REASON' results/day/hg38/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ/align/sent/dmd/snv/sentd/contam_identity/read_haps/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.sent.dmd.sentd.read_haps.txt
4194-
4195:        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
4196-    cluster_jobid: 3385
4197-
4198-Error executing rule read_haps_contam_identity on cluster (jobid: 84, external: 3385, jobscript: /fsx/analysis_results/ubuntu/ccv20260529r40_illumina_hg002_kitchensink_multiqc/daylily-omics-analysis/.snakemake/tmp.yun4xh_6/snakejob.read_haps_contam_identity.84.sh). For error details see the cluster log and the log files of the involved rule(s).
4199-[Sat May 30 10:40:45 2026]
4200-Finished job 57.
4201-92 of 161 steps (57%) done
4202-[Sat May 30 10:40:46 2026]
4203-Finished job 63.
4204-93 of 161 steps (58%) done
4205-[Sat May 30 10:40:56 2026]
4206-Finished job 43.
4207-94 of 161 steps (58%) done
4208-[Sat May 30 10:40:56 2026]
4209-Finished job 35.
4210-95 of 161 steps (59%) done
4211-[Sat May 30 10:40:56 2026]
4212-Finished job 45.
4213-96 of 161 steps (60%) done
4214-[Sat May 30 10:40:56 2026]
4215-Finished job 53.
4216-97 of 161 steps (60%) done
4217-[Sat May 30 10:40:56 2026]
4218-Finished job 27.
4219-98 of 161 steps (61%) done
4220-[Sat May 30 10:40:56 2026]
=== PATTERN non-zero exit status ===
=== PATTERN MissingOutputException ===
=== PATTERN FailedRuleException ===
=== PATTERN Complete log ===
=== last 260 lines ===
138 of 161 steps (86%) done
[Sat May 30 10:41:59 2026]
Finished job 149.
139 of 161 steps (86%) done
[Sat May 30 10:43:29 2026]
Finished job 113.
140 of 161 steps (87%) done
[Sat May 30 10:45:00 2026]
Finished job 80.
141 of 161 steps (88%) done
Select jobs to execute...

[Sat May 30 10:45:00 2026]
rule relatedness_batch_somalier_relate:
    input: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/extract/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.somalier
    output: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.samples.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.html
    log: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/logs/cohort.log
    jobid: 79
    reason: Missing output files: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.samples.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv; Input files updated by another job: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/extract/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.somalier
    wildcards: alnr=sent, ddup=dmd
    threads: 4
    resources: mem_mb=3000, mem_mib=2862, disk_mb=1000, disk_mib=954, tmpdir=<TBD>, threads=1, time=5440, partition=i192,i192mem,i128, vcpu=4, distribution=block, exclusive=, constraint=, exclude=, include=


        set -euo pipefail
        mkdir -p $(dirname results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv) $(dirname results/day/hg38/other_reports/relatedness/sent/dmd/somalier/logs/cohort.log)
        rm -f results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.samples.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.html
        if [ 1 -lt 2 ]; then
            printf "sample_a\tsample_b\trelatedness\tibs0\n" > results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv
            printf "sample_id\tgroup\n" > results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.samples.tsv
            printf "sample_id\tgroup\n" > results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv
            printf "<html><body><h1>Relatedness QC</h1><p>No pairs available.</p></body></html>\n" > results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.html
        else
            somalier relate results/day/hg38/other_reports/relatedness/sent/dmd/somalier/extract/JEMILMN0P1-HG002-0p1x-1-D0-PF-ILMN-NOVASEQ.somalier -o results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort > results/day/hg38/other_reports/relatedness/sent/dmd/somalier/logs/cohort.log 2>&1
        fi
        for expected_output in results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.samples.tsv results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.html; do
            if [[ ! -s "$expected_output" ]]; then
                printf 'ERROR: somalier relate did not create declared cohort output: %s\n' "$expected_output" >> results/day/hg38/other_reports/relatedness/sent/dmd/somalier/logs/cohort.log
                exit 1
            fi
        done

Submitted job 79 with external jobid '3421'.
[Sat May 30 10:45:20 2026]
Finished job 79.
142 of 161 steps (88%) done
Select jobs to execute...

[Sat May 30 10:45:20 2026]
localrule relatedness_batch_report:
    input: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_manifest.tsv
    output: results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_pairs_classified.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_summary.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_report.html
    jobid: 78
    reason: Missing output files: results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_summary.tsv; Input files updated by another job: results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.groups.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/somalier/cohort.pairs.tsv, results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_manifest.tsv
    wildcards: alnr=sent, ddup=dmd
    resources: mem_mb=3000, mem_mib=2862, disk_mb=1000, disk_mib=954, tmpdir=/dev/shm, threads=1, time=5440, partition=i192,i128,i192mem, vcpu=1, distribution=block, exclusive=, constraint=, exclude=, include=

python -c "from __future__ import print_function; import sys, json; print(json.dumps([sys.version_info.major, sys.version_info.minor]))"
Activating conda environment: ../../../../resources/environments/conda/ubuntu/ip-10-0-0-88/fb0da2b0be44b336d1294fa52fffc58e_
python /fsx/analysis_results/ubuntu/ccv20260529r40_illumina_hg002_kitchensink_multiqc/daylily-omics-analysis/.snakemake/scripts/tmpn1daczhf.relatedness_report.py
Activating conda environment: ../../../../resources/environments/conda/ubuntu/ip-10-0-0-88/fb0da2b0be44b336d1294fa52fffc58e_
[Sat May 30 10:45:21 2026]
Finished job 78.
143 of 161 steps (89%) done
Select jobs to execute...

[Sat May 30 10:45:21 2026]
localrule relatedness_batch_gather:
    input: results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_summary.tsv
    output: results/day/hg38/other_reports/relatedness_mqc.tsv
    jobid: 77
    reason: Missing output files: results/day/hg38/other_reports/relatedness_mqc.tsv; Input files updated by another job: results/day/hg38/other_reports/relatedness/sent/dmd/relatedness_summary.tsv
    resources: mem_mb=3000, mem_mib=2862, disk_mb=1000, disk_mib=954, tmpdir=/dev/shm, threads=1, time=5440, partition=i192,i128,i192mem, vcpu=1, distribution=block, exclusive=, constraint=, exclude=, include=

[Sat May 30 10:45:24 2026]
Finished job 77.
144 of 161 steps (89%) done
Select jobs to execute...

[Sat May 30 10:45:24 2026]
loc--output truncated--