Discovering test cases in: /home/bulat/tesis/citic/alecci/concurrency_validation/alecci_benchmarks/SCTBench
Found 48 test case(s)
Running with 24 parallel job(s)
  [1/48] account-no.yaml/SCTBench/account-no... PASS
  [2/48] account_bad-no.yaml/SCTBench/account_bad-no... PASS
  [3/48] arithmetic_prog-no.yaml/SCTBench/arithmetic_prog-no... PASS
  [4/48] arithmetic_prog_bad-no.yaml/SCTBench/arithmetic_prog_bad-no... PASS
  [5/48] carter01_bad-no.yaml/SCTBench/carter01_bad-no... PASS
  [6/48] circular-buffer-flag-no.yaml/SCTBench/circular-buffer-flag-no... MISS
  [7/48] circular-buffer-flag-ok-no.yaml/SCTBench/circular-buffer-flag-ok-no... PASS
  [8/48] deadlock01_bad-no.yaml/SCTBench/deadlock01_bad-no... PASS
  [9/48] din_phil2_sat-yes.yaml/SCTBench/din_phil2_sat-yes... PASS
  [10/48] din_phil2_unsat-no.yaml/SCTBench/din_phil2_unsat-no... UNEXPECTED
  [11/48] din_phil3_sat-yes.yaml/SCTBench/din_phil3_sat-yes... PASS
  [12/48] din_phil3_unsat-no.yaml/SCTBench/din_phil3_unsat-no... UNEXPECTED
  [13/48] din_phil4_sat-yes.yaml/SCTBench/din_phil4_sat-yes... PASS
  [14/48] din_phil4_unsat-no.yaml/SCTBench/din_phil4_unsat-no... UNEXPECTED
  [15/48] din_phil5_sat-yes.yaml/SCTBench/din_phil5_sat-yes... PASS
  [16/48] din_phil5_unsat-no.yaml/SCTBench/din_phil5_unsat-no... UNEXPECTED
  [17/48] din_phil6_sat-yes.yaml/SCTBench/din_phil6_sat-yes... PASS
  [18/48] din_phil6_unsat-no.yaml/SCTBench/din_phil6_unsat-no... UNEXPECTED
  [19/48] din_phil7_sat-yes.yaml/SCTBench/din_phil7_sat-yes... PASS
  [20/48] din_phil7_unsat-no.yaml/SCTBench/din_phil7_unsat-no... UNEXPECTED
  [21/48] dpor_example1-no.yaml/SCTBench/dpor_example1-no... PASS
  [22/48] dpor_example3-yes.yaml/SCTBench/dpor_example3-yes... PASS
  [23/48] lazy01-no.yaml/SCTBench/lazy01-no... MISS
  [24/48] micro_unprotected_10-yes.yaml/SCTBench/micro_unprotected_10-yes... PASS
  [25/48] micro_unprotected_2-yes.yaml/SCTBench/micro_unprotected_2-yes... PASS
  [26/48] micro_unprotected_3-yes.yaml/SCTBench/micro_unprotected_3-yes... PASS
  [27/48] phase01-no.yaml/SCTBench/phase01-no... PASS
  [28/48] phase01_bad-no.yaml/SCTBench/phase01_bad-no... PASS
  [29/48] queue-flag-no.yaml/SCTBench/queue-flag-no... MISS
  [30/48] race01-yes.yaml/SCTBench/race01-yes... PASS
  [31/48] reorder_10-yes.yaml/SCTBench/reorder_10-yes... PASS
  [32/48] reorder_20-yes.yaml/SCTBench/reorder_20-yes... PASS
  [33/48] reorder_3-yes.yaml/SCTBench/reorder_3-yes... PASS
  [34/48] reorder_4-yes.yaml/SCTBench/reorder_4-yes... PASS
  [35/48] reorder_5-yes.yaml/SCTBench/reorder_5-yes... PASS
  [36/48] simple_mutex-no.yaml/SCTBench/simple_mutex-no... PASS
  [37/48] stack-no.yaml/SCTBench/stack-no... MISS
  [38/48] stateful01-no.yaml/SCTBench/stateful01-no... PASS
  [39/48] sync01-no.yaml/SCTBench/sync01-no... PASS
  [40/48] sync01_bad-no.yaml/SCTBench/sync01_bad-no... PASS
  [41/48] sync01_inspect-no.yaml/SCTBench/sync01_inspect-no... PASS
  [42/48] sync02-no.yaml/SCTBench/sync02-no... PASS
  [43/48] sync02_bad-no.yaml/SCTBench/sync02_bad-no... PASS
  [44/48] token_ring_bad-no.yaml/SCTBench/token_ring_bad-no... PASS
  [45/48] twostage-no.yaml/SCTBench/twostage-no... MISS
  [46/48] twostage_100-no.yaml/SCTBench/twostage_100-no... MISS
  [47/48] wronglock-yes.yaml/SCTBench/wronglock-yes... PASS
  [48/48] wronglock_3-yes.yaml/SCTBench/wronglock_3-yes... PASS
Results written to:           concurrency_validation/results/sct_results.csv
Issue breakdown written to:   concurrency_validation/results/issue_breakdown_sct_results.csv
Confusion matrix written to:   concurrency_validation/results/sct_confusion_matrix.tex

============================================================
TEST SUMMARY
============================================================
Total:          48
Compile failed: 0

Buggy programs   (expected: bug):   32
  Detected [PASS]                           26
  Missed   [MISS]                            6
Correct programs (expected: none):  16
  Clean            [PASS]                   10
  False positives  [UNEXPECTED]              6
============================================================
Discovering test cases in: /home/bulat/tesis/citic/alecci/concurrency_validation/alecci_benchmarks/labeled_benchmarks
Found 212 test case(s)
Running with 24 parallel job(s)
  [1/212] ClassExamples/build_h2o/0001... PASS
  [2/212] ClassExamples/build_h2o/0002... PASS
  [3/212] ClassExamples/build_h2o/0003... PASS
  [4/212] ClassExamples/build_h2o/0005... PASS
  [5/212] ClassExamples/build_h2o/0006... PASS
  [6/212] ClassExamples/build_h2o/0007... PASS
  [7/212] ClassExamples/build_h2o/0008... PASS
  [8/212] ClassExamples/build_h2o/0009... PASS
  [9/212] ClassExamples/build_h2o/0035... MISS
  [10/212] ClassExamples/dancing_pairs/0001... PASS
  [11/212] ClassExamples/dancing_pairs/0002... PASS
  [12/212] ClassExamples/dancing_pairs/0003... PASS
  [13/212] ClassExamples/dancing_pairs/0004... PASS
  [14/212] ClassExamples/dancing_pairs/0005... PASS
  [15/212] ClassExamples/dancing_pairs/0006... PASS
  [16/212] ClassExamples/dancing_pairs/0007... PASS
  [17/212] ClassExamples/dancing_pairs/0008... PASS
  [18/212] ClassExamples/dining_philosophers/0001... PASS
  [19/212] ClassExamples/dining_philosophers/0002... PASS
  [20/212] ClassExamples/dining_philosophers/0003... MISS
  [21/212] ClassExamples/dining_philosophers/0004... PASS
  [22/212] ClassExamples/dining_philosophers/0005... MISS
  [23/212] ClassExamples/network_simul_packet_loss_fix/0001... PASS
  [24/212] ClassExamples/network_simul_packet_loss_fix/0002... PASS
  [25/212] ClassExamples/network_simul_packet_loss_fix/0003... PASS
  [26/212] ClassExamples/network_simul_packet_loss_fix/0004... MISS
  [27/212] ClassExamples/network_simul_packet_loss_fix/0005... PASS
  [28/212] ClassExamples/prod_cons_bound/0001... PASS
  [29/212] ClassExamples/prod_cons_bound/0002... PASS
  [30/212] ClassExamples/prod_cons_bound/0003... PASS
  [31/212] ClassExamples/prod_cons_bound/0004... MISS
  [32/212] ClassExamples/prod_cons_bound/0005... PASS
  [33/212] ClassExamples/prod_cons_queue/0001... PASS
  [34/212] ClassExamples/prod_cons_queue/0002... PASS
  [35/212] ClassExamples/prod_cons_queue/0003... MISS
  [36/212] ClassExamples/prod_cons_queue/0004... PASS
  [37/212] ClassExamples/prod_cons_queue/0005... MISS
  [38/212] ClassExamples/prod_cons_unbound/0001... PASS
  [39/212] ClassExamples/prod_cons_unbound/0002... MISS
  [40/212] ClassExamples/prod_cons_unbound/0003... PASS
  [41/212] ClassExamples/prod_cons_unbound/0004... MISS
  [42/212] ClassExamples/prod_cons_unbound/0005... PASS
  [43/212] ClassExamples/relay_race/0001... PASS
  [44/212] ClassExamples/relay_race/0002... PASS
  [45/212] ClassExamples/relay_race/0003... PASS
  [46/212] ClassExamples/relay_race/0004... PASS
  [47/212] ClassExamples/relay_race/0005... PASS
  [48/212] ClassExamples/taskc_patterns/0001... PASS
  [49/212] ClassExamples/taskc_patterns/0002... PASS
  [50/212] ClassExamples/taskc_patterns/0003... PASS
  [51/212] ClassExamples/taskc_patterns/0004... PASS
  [52/212] ClassExamples/taskc_patterns/0005... PASS
  [53/212] ClassExamples/taskc_patterns/0006... PASS
  [54/212] ClassExamples/taskc_patterns/0007... PASS
  [55/212] ClassExamples/taskc_patterns/0008... PASS
  [56/212] ClassExamples/taskc_patterns/0009... PASS
  [57/212] DataRaceBench/N1/DRB045-doall1-orig-no... PASS
  [58/212] DataRaceBench/N1/DRB046-doall2-orig-no... PASS
  [59/212] DataRaceBench/N1/DRB047-doallchar-orig-no... PASS
  [60/212] DataRaceBench/N1/DRB053-inneronly1-orig-no... PASS
  [61/212] DataRaceBench/N1/DRB054-inneronly2-orig-no... PASS
  [62/212] DataRaceBench/N1/DRB077-single-orig-no... PASS
  [63/212] DataRaceBench/N1/DRB103-master-orig-no... PASS
  [64/212] DataRaceBench/N1/DRB113-default-orig-no... PASS
  [65/212] DataRaceBench/N2/DRB048-firstprivate-orig-no... PASS
  [66/212] DataRaceBench/N2/DRB060-matrixmultiply-orig-no... PASS
  [67/212] DataRaceBench/N2/DRB061-matrixvector1-orig-no... PASS
  [68/212] DataRaceBench/N2/DRB062-matrixvector2-orig-no... PASS
  [69/212] DataRaceBench/N2/DRB063-outeronly1-orig-no... PASS
  [70/212] DataRaceBench/N2/DRB064-outeronly2-orig-no... PASS
  [71/212] DataRaceBench/N2/DRB093-doall2-collapse-orig-no... PASS
  [72/212] DataRaceBench/N2/DRB094-doall2-ordered-orig-no... PASS
  [73/212] DataRaceBench/N2/DRB121-reduction-orig-no... PASS
  [74/212] DataRaceBench/N2/DRB170-nestedloops-orig-no... PASS
  [75/212] DataRaceBench/N3/DRB108-atomic-orig-no... PASS
  [76/212] DataRaceBench/N3/DRB120-barrier-orig-no... PASS
  [77/212] DataRaceBench/N3/DRB141-reduction-barrier-orig-no... PASS
  [78/212] DataRaceBench/N3/DRB172-critical2-orig-no... PASS
  [79/212] DataRaceBench/N3/DRB190-critical-section2-no... PASS
  [80/212] DataRaceBench/N7/DRB057-jacobiinitialize-orig-no... PASS
  [81/212] DataRaceBench/N7/DRB058-jacobikernel-orig-no... PASS
  [82/212] DataRaceBench/N7/DRB065-pireduction-orig-no... PASS
  [83/212] DataRaceBench/Y1/DRB001-antidep1-orig-yes... PASS
  [84/212] DataRaceBench/Y1/DRB002-antidep1-var-yes... PASS
  [85/212] DataRaceBench/Y1/DRB003-antidep2-orig-yes... PASS
  [86/212] DataRaceBench/Y1/DRB004-antidep2-var-yes... PASS
  [87/212] DataRaceBench/Y1/DRB016-outputdep-orig-yes... PASS
  [88/212] DataRaceBench/Y1/DRB017-outputdep-var-yes... PASS
  [89/212] DataRaceBench/Y1/DRB018-plusplus-orig-yes... PASS
  [90/212] DataRaceBench/Y1/DRB019-plusplus-var-yes... PASS
  [91/212] DataRaceBench/Y1/DRB029-truedep1-orig-yes... PASS
  [92/212] DataRaceBench/Y1/DRB030-truedep1-var-yes... PASS
  [93/212] DataRaceBench/Y1/DRB031-truedepfirstdimension-orig-yes... PASS
  [94/212] DataRaceBench/Y1/DRB032-truedepfirstdimension-var-yes... PASS
  [95/212] DataRaceBench/Y1/DRB033-truedeplinear-orig-yes... PASS
  [96/212] DataRaceBench/Y1/DRB034-truedeplinear-var-yes... PASS
  [97/212] DataRaceBench/Y1/DRB035-truedepscalar-orig-yes... PASS
  [98/212] DataRaceBench/Y1/DRB036-truedepscalar-var-yes... PASS
  [99/212] DataRaceBench/Y1/DRB037-truedepseconddimension-orig-yes... PASS
  [100/212] DataRaceBench/Y1/DRB038-truedepseconddimension-var-yes... PASS
  [101/212] DataRaceBench/Y1/DRB039-truedepsingleelement-orig-yes... PASS
  [102/212] DataRaceBench/Y1/DRB040-truedepsingleelement-var-yes... PASS
  [103/212] DataRaceBench/Y1/DRB075-getthreadnum-orig-yes... PASS
  [104/212] DataRaceBench/Y1/DRB169-missingsyncwrite-orig-yes... PASS
  [105/212] DataRaceBench/Y2/DRB009-lastprivatemissing-orig-yes... PASS
  [106/212] DataRaceBench/Y2/DRB010-lastprivatemissing-var-yes... PASS
  [107/212] DataRaceBench/Y2/DRB020-privatemissing-orig-yes... PASS
  [108/212] DataRaceBench/Y2/DRB021-reductionmissing-orig-yes... PASS
  [109/212] DataRaceBench/Y2/DRB022-reductionmissing-var-yes... PASS
  [110/212] DataRaceBench/Y2/DRB023-reductionmissing-var-yes... PASS
  [111/212] DataRaceBench/Y2/DRB028-privatemissing-orig-yes... PASS
  [112/212] DataRaceBench/Y2/DRB111-linearmissing-orig-yes... PASS
  [113/212] DataRaceBench/Y2/DRB191-critical-section2-yes... PASS
  [114/212] DataRaceBench/Y3/DRB011-minusminus-orig-yes... PASS
  [115/212] DataRaceBench/Y3/DRB012-minusminus-var-yes... PASS
  [116/212] DataRaceBench/Y3/DRB140-reduction-barrier-orig-yes... MISS
  [117/212] DeepRace/with_datarace/array-max-nolock2-yes... PASS
  [118/212] DeepRace/with_datarace/array-total-nolock-yes... PASS
  [119/212] DeepRace/with_datarace/count-incdec-nolock-yes... PASS
  [120/212] DeepRace/with_datarace/fetch-increment-nolock-yes... PASS
  [121/212] DeepRace/with_datarace/fib-array-race-yes... PASS
  [122/212] DeepRace/with_datarace/firstprivate-sum-nolock-yes... PASS
  [123/212] DeepRace/with_datarace/float-pi-nolock-yes... PASS
  [124/212] DeepRace/with_datarace/float-pi-worksharing-nolock-yes... PASS
  [125/212] DeepRace/with_datarace/float-pi2-nolock-yes... PASS
  [126/212] DeepRace/with_datarace/for-sum-nolock-yes... PASS
  [127/212] DeepRace/with_datarace/global-max-nolock-yes... PASS
  [128/212] DeepRace/with_datarace/hitung-nolock-yes... PASS
  [129/212] DeepRace/with_datarace/kmeans-race-yes... PASS
  [130/212] DeepRace/with_datarace/matrix-mul-k-race-yes... PASS
  [131/212] DeepRace/with_datarace/matrix-sum-nolock-yes... PASS
  [132/212] DeepRace/with_datarace/monte-carlo-nolock-yes... PASS
  [133/212] DeepRace/with_datarace/parallel-increment-yes... PASS
  [134/212] DeepRace/with_datarace/parallel-max-nolock-yes... PASS
  [135/212] DeepRace/with_datarace/shared-a-pfor-yes... PASS
  [136/212] DeepRace/with_datarace/shared-i-nolock-yes... PASS
  [137/212] DeepRace/with_datarace/shared-increment-nolock-yes... PASS
  [138/212] DeepRace/with_datarace/shared-local-accumulate-yes... PASS
  [139/212] DeepRace/with_datarace/shared-threadnum-nolock-yes... PASS
  [140/212] DeepRace/with_datarace/sum-private-nolock-yes... PASS
  [141/212] DeepRace/without_datarace/array-max-critical2-no... PASS
  [142/212] DeepRace/without_datarace/combined-array-count-no... PASS
  [143/212] DeepRace/without_datarace/critical-id-no... PASS
  [144/212] DeepRace/without_datarace/determinant-no... PASS
  [145/212] DeepRace/without_datarace/fibonacci-critical-no... PASS
  [146/212] DeepRace/without_datarace/float-pi-cyclic-no... PASS
  [147/212] DeepRace/without_datarace/float-pi-inline-no... PASS
  [148/212] DeepRace/without_datarace/float-pi-padded-no... PASS
  [149/212] DeepRace/without_datarace/float-pi-partial-no... PASS
  [150/212] DeepRace/without_datarace/float-pi-syncintegrate-no... PASS
  [151/212] DeepRace/without_datarace/global-max-mutex-no... PASS
  [152/212] DeepRace/without_datarace/hello-atomic-no... PASS
  [153/212] DeepRace/without_datarace/histogramaSerie_-_main... PASS
  [154/212] DeepRace/without_datarace/min-max-avg-mutex-no... PASS
  [155/212] DeepRace/without_datarace/parallel-increment-atomic-no... PASS
  [156/212] DeepRace/without_datarace/parallel-max-atomic-no... PASS
  [157/212] DeepRace/without_datarace/pi-integral-mutex-no... PASS
  [158/212] DeepRace/without_datarace/product-mutex-no... PASS
  [159/212] DeepRace/without_datarace/reduction-mutex-no... PASS
  [160/212] DeepRace/without_datarace/shared-local-accumulate-no... PASS
  [161/212] DeepRace/with_datarace/float-pi-noreduction-yes... PASS
  [162/212] DeepRace/with_datarace/isum-nolock-yes... PASS
  [163/212] DeepRace/with_datarace/matrix-add-shared-yes... PASS
  [164/212] DeepRace/with_datarace/shared-myid-yes... PASS
  [165/212] DeepRace/with_datarace/shared-tid-pfor-yes... PASS
  [166/212] DeepRace/with_datarace/shared-tmp-yes... PASS
  [167/212] DeepRace/with_datarace/shared-x-nolock2-yes... PASS
  [168/212] DeepRace/with_datarace/shared-x-pfor-yes... PASS
  [169/212] DeepRace/with_datarace/shared-x-yes... PASS
  [170/212] DeepRace/without_datarace/18_for_wait_-_main... PASS
  [171/212] DeepRace/without_datarace/array-add-pfor-no... PASS
  [172/212] DeepRace/without_datarace/array-fill-tid-no... PASS
  [173/212] DeepRace/without_datarace/array-negate-no... PASS
  [174/212] DeepRace/without_datarace/barrier-y-z-no... PASS
  [175/212] DeepRace/without_datarace/digit-square-pow-no... PASS
  [176/212] DeepRace/without_datarace/local-max-per-thread-no... PASS
  [177/212] DeepRace/without_datarace/local-myid-no... PASS
  [178/212] DeepRace/without_datarace/local-myid-pfor-no... PASS
  [179/212] DeepRace/without_datarace/local-tmp-no... PASS
  [180/212] DeepRace/without_datarace/local-vars-no... PASS
  [181/212] DeepRace/without_datarace/matrix2loops_-_main... PASS
  [182/212] DeepRace/without_datarace/matrix_part1_-_main... PASS
  [183/212] DeepRace/without_datarace/two-pfor-fill-no... PASS
  [184/212] DeepRace/with_datarace/bucket-sum-mutex-no... MISS
  [185/212] DeepRace/with_datarace/canibais-cond-yes... PASS
  [186/212] DeepRace/with_datarace/count-threshold-cond-yes... PASS
  [187/212] DeepRace/with_datarace/counter-nolock-yes... PASS
  [188/212] DeepRace/with_datarace/flag-nolock-yes... PASS
  [189/212] DeepRace/with_datarace/global-sum-nolock-yes... PASS
  [190/212] DeepRace/with_datarace/histogram-nolock-yes... PASS
  [191/212] DeepRace/with_datarace/histogram-rand-no... MISS
  [192/212] DeepRace/with_datarace/readwrite-cond-yes... PASS
  [193/212] DeepRace/with_datarace/ticket-counter-no... MISS
  [194/212] DeepRace/with_datarace/two-thread-flag-cond-no... MISS
  [195/212] DeepRace/with_datarace/unisex-bathroom-cond-yes... PASS
  [196/212] DeepRace/with_datarace/wait-all-cond-yes... MISS+UNEXPECTED
  [197/212] DeepRace/without_datarace/array-sum-mutex-no... PASS
  [198/212] DeepRace/without_datarace/bounded-buffer-cond-no... PASS
  [199/212] DeepRace/without_datarace/circular-buffer-cond-no... PASS
  [200/212] DeepRace/without_datarace/count-threshold-4t-cond-no... PASS
  [201/212] DeepRace/without_datarace/count-threshold-cond-no... PASS
  [202/212] DeepRace/without_datarace/count-threshold-outer-cond-yes... PASS
  [203/212] DeepRace/without_datarace/counter-mutex-no... PASS
  [204/212] DeepRace/without_datarace/counter-mutex2-no... PASS
  [205/212] DeepRace/without_datarace/counter-mutex3-no... PASS
  [206/212] DeepRace/without_datarace/counter-mutex4-no... PASS
  [207/212] DeepRace/without_datarace/flag-signal-cond-yes... PASS
  [208/212] DeepRace/without_datarace/global-sum-mutex-no... PASS
  [209/212] DeepRace/without_datarace/lock-order-no... UNEXPECTED
  [210/212] DeepRace/without_datarace/lock-order-rand-no... UNEXPECTED
  [211/212] DeepRace/without_datarace/ping-pong-cond-no... PASS
  [212/212] DeepRace/without_datarace/scatter-add-mutex-no... PASS
Results written to:           concurrency_validation/results/labeled_results.csv
Issue breakdown written to:   concurrency_validation/results/issue_breakdown_labeled_results.csv
LaTeX table written to:       concurrency_validation/results/labeled_table.tex
Confusion matrix written to:   concurrency_validation/results/labeled_confusion_matrix.tex

============================================================
TEST SUMMARY
============================================================
Total:          212
Compile failed: 0

Detection outcomes:
  All expected found, none unexpected [PASS]    195
  All expected found + extra detected  [UNEXPECTED] 2
  Missed expected issues               [MISS]   14
  Missed expected + extras detected    [MISS+UNEXPECTED] 1

Breakdown by concurrency issue type:
  Issue type           Expected  Detected  Missed  Unexp.
  -------------------  --------  --------  ------  ------
  data_race                 104        96       8       0
  deadlock                   10         6       6       2
  mutex_destruction           1         1       0       0
  thread_leak                24        23       2       1
============================================================
