Initializing analyzer...
Using model: all-MiniLM-L6-v2
Device: auto

================================================================================
Analyzing: /Users/caevans/repos/loghub/Spark/Spark_2k.log
Total lines: 2,000
================================================================================

Analysis Statistics:
  Total windows created: 400
  Significant windows: 40
  Merged blocks: 22
  Processing time: 2.98s

Score Distribution:
  Min:    0.0042
  Mean:   0.0349
  Median: 0.0231
  P90:    0.0615
  Max:    0.4564

                               Significant Blocks
================================================================================
<block lines="1-25" score="0.4564">
17/06/09 20:10:40 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
17/06/09 20:10:40 INFO spark.SecurityManager: Changing view acls to: yarn,curi
17/06/09 20:10:40 INFO spark.SecurityManager: Changing modify acls to: yarn,curi
17/06/09 20:10:40 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, curi); users with modify permissions: Set(yarn, curi)
17/06/09 20:10:41 INFO spark.SecurityManager: Changing view acls to: yarn,curi
17/06/09 20:10:41 INFO spark.SecurityManager: Changing modify acls to: yarn,curi
17/06/09 20:10:41 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, curi); users with modify permissions: Set(yarn, curi)
17/06/09 20:10:41 INFO slf4j.Slf4jLogger: Slf4jLogger started
17/06/09 20:10:41 INFO Remoting: Starting remoting
17/06/09 20:10:41 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutorActorSystem@mesos-slave-07:55904]
17/06/09 20:10:41 INFO util.Utils: Successfully started service 'sparkExecutorActorSystem' on port 55904.
17/06/09 20:10:41 INFO storage.DiskBlockManager: Created local directory at /opt/hdfs/nodemanager/usercache/curi/appcache/application_1485248649253_0147/blockmgr-70293f72-844a-4b39-9ad6-fb0ad7e364e4
17/06/09 20:10:41 INFO storage.MemoryStore: MemoryStore started with capacity 17.7 GB
17/06/09 20:10:42 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.10.34.11:48069
17/06/09 20:10:42 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver
17/06/09 20:10:42 INFO executor.Executor: Starting executor ID 5 on host mesos-slave-07
17/06/09 20:10:42 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 40984.
17/06/09 20:10:42 INFO netty.NettyBlockTransferService: Server created on 40984
17/06/09 20:10:42 INFO storage.BlockManagerMaster: Trying to register BlockManager
17/06/09 20:10:42 INFO storage.BlockManagerMaster: Registered BlockManager
17/06/09 20:10:45 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 0
17/06/09 20:10:45 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1
17/06/09 20:10:45 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 2
17/06/09 20:10:45 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 3
17/06/09 20:10:45 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)
</block>

<block lines="41-80" score="0.2969">
17/06/09 20:10:46 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:14584+7292
17/06/09 20:10:46 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:0+7292
17/06/09 20:10:46 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:7292+7292
17/06/09 20:10:46 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:29168+7292
17/06/09 20:10:46 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8
17/06/09 20:10:46 INFO storage.MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 21.4 KB, free 35.4 KB)
17/06/09 20:10:46 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8 took 19 ms
17/06/09 20:10:46 INFO storage.MemoryStore: Block broadcast_8 stored as values in memory (estimated size 281.6 KB, free 317.0 KB)
17/06/09 20:10:47 INFO Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
17/06/09 20:10:47 INFO Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
17/06/09 20:10:47 INFO Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
17/06/09 20:10:47 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
17/06/09 20:10:47 INFO Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 0
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 93.0 B, free 317.1 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Reading broadcast variable 0 took 21 ms
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 384.0 B, free 317.5 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 4
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 95.0 B, free 317.6 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Reading broadcast variable 4 took 18 ms
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 384.0 B, free 318.0 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 3
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 152.0 B, free 318.1 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Reading broadcast variable 3 took 17 ms
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 384.0 B, free 318.5 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 1
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 107.0 B, free 318.6 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Reading broadcast variable 1 took 16 ms
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 384.0 B, free 319.0 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 2
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 93.0 B, free 319.0 KB)
17/06/09 20:10:48 INFO broadcast.TorrentBroadcast: Reading broadcast variable 2 took 16 ms
17/06/09 20:10:48 INFO storage.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 384.0 B, free 319.4 KB)
17/06/09 20:10:48 INFO python.PythonRunner: Times: total = 1072, boot = 856, init = 210, finish = 6
17/06/09 20:10:48 INFO python.PythonRunner: Times: total = 1114, boot = 885, init = 223, finish = 6
17/06/09 20:10:48 INFO python.PythonRunner: Times: total = 1074, boot = 869, init = 199, finish = 6
17/06/09 20:10:48 INFO python.PythonRunner: Times: total = 1078, boot = 851, init = 219, finish = 8
17/06/09 20:10:48 INFO python.PythonRunner: Times: total = 1077, boot = 865, init = 206, finish = 6
17/06/09 20:10:48 INFO storage.MemoryStore: Block rdd_2_2 stored as bytes in memory (estimated size 850.0 B, free 320.3 KB)
17/06/09 20:10:48 INFO storage.MemoryStore: Block rdd_2_3 stored as bytes in memory (estimated size 930.0 B, free 321.2 KB)
</block>

<block lines="166-175" score="0.0746">
17/06/09 20:10:53 INFO spark.CacheManager: Partition rdd_6_4 not found, computing it
17/06/09 20:10:53 INFO spark.CacheManager: Partition rdd_6_3 not found, computing it
17/06/09 20:10:53 INFO spark.CacheManager: Partition rdd_6_0 not found, computing it
17/06/09 20:10:53 INFO spark.CacheManager: Partition rdd_6_1 not found, computing it
17/06/09 20:10:53 INFO spark.CacheManager: Partition rdd_6_2 not found, computing it
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_2_4 locally
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_2_3 locally
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_2_1 locally
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_2_0 locally
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_2_2 locally
</block>

<block lines="201-210" score="0.0746">
17/06/09 20:10:53 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 195
17/06/09 20:10:53 INFO executor.Executor: Running task 2.0 in stage 4.0 (TID 195)
17/06/09 20:10:53 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 198
17/06/09 20:10:53 INFO executor.Executor: Running task 3.0 in stage 4.0 (TID 198)
17/06/09 20:10:53 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 199
17/06/09 20:10:53 INFO executor.Executor: Running task 4.0 in stage 4.0 (TID 199)
17/06/09 20:10:53 INFO storage.MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 5.7 KB, free 376.4 KB)
17/06/09 20:10:53 INFO broadcast.TorrentBroadcast: Reading broadcast variable 13 took 41 ms
17/06/09 20:10:53 INFO storage.MemoryStore: Block broadcast_13 stored as values in memory (estimated size 10.1 KB, free 386.5 KB)
17/06/09 20:10:53 INFO storage.BlockManager: Found block rdd_6_0 locally
</block>

<block lines="436-445" score="0.0616">
17/06/09 20:10:54 INFO storage.BlockManager: Found block rdd_2_2 locally
17/06/09 20:10:54 INFO storage.BlockManager: Found block rdd_2_1 locally
17/06/09 20:10:54 INFO python.PythonRunner: Times: total = 40, boot = -95, init = 135, finish = 0
17/06/09 20:10:54 INFO python.PythonRunner: Times: total = 39, boot = -96, init = 135, finish = 0
17/06/09 20:10:54 INFO python.PythonRunner: Times: total = 42, boot = -95, init = 137, finish = 0
17/06/09 20:10:54 INFO python.PythonRunner: Times: total = 42, boot = -97, init = 139, finish = 0
17/06/09 20:10:54 INFO python.PythonRunner: Times: total = 43, boot = -85, init = 127, finish = 1
17/06/09 20:10:54 INFO storage.MemoryStore: Block rdd_16_4 stored as bytes in memory (estimated size 263.0 B, free 386.4 KB)
17/06/09 20:10:54 INFO storage.MemoryStore: Block rdd_16_0 stored as bytes in memory (estimated size 191.0 B, free 386.6 KB)
17/06/09 20:10:54 INFO storage.MemoryStore: Block rdd_16_3 stored as bytes in memory (estimated size 232.0 B, free 386.8 KB)
</block>

<block lines="691-700" score="0.0632">
17/06/09 20:10:56 INFO spark.CacheManager: Partition rdd_26_4 not found, computing it
17/06/09 20:10:56 INFO spark.CacheManager: Partition rdd_26_3 not found, computing it
17/06/09 20:10:56 INFO spark.CacheManager: Partition rdd_26_2 not found, computing it
17/06/09 20:10:56 INFO spark.CacheManager: Partition rdd_26_1 not found, computing it
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_0 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_4 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_3 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_2 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_1 locally
17/06/09 20:10:56 INFO python.PythonRunner: Times: total = 40, boot = -79, init = 119, finish = 0
</block>

<block lines="796-805" score="0.0651">
17/06/09 20:10:56 INFO spark.CacheManager: Partition rdd_30_1 not found, computing it
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_0 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_4 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_3 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_2 locally
17/06/09 20:10:56 INFO storage.BlockManager: Found block rdd_2_1 locally
17/06/09 20:10:56 INFO python.PythonRunner: Times: total = 37, boot = -88, init = 125, finish = 0
17/06/09 20:10:56 INFO python.PythonRunner: Times: total = 39, boot = -86, init = 125, finish = 0
17/06/09 20:10:56 INFO storage.MemoryStore: Block rdd_30_0 stored as bytes in memory (estimated size 324.0 B, free 405.4 KB)
17/06/09 20:10:56 INFO python.PythonRunner: Times: total = 39, boot = -86, init = 125, finish = 0
</block>

<block lines="821-830" score="0.0820">
17/06/09 20:10:56 INFO executor.Executor: Finished task 3.0 in stage 22.0 (TID 918). 2667 bytes result sent to driver
17/06/09 20:10:56 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 926
17/06/09 20:10:56 INFO executor.Executor: Running task 0.0 in stage 23.0 (TID 926)
17/06/09 20:10:56 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 940
17/06/09 20:10:56 INFO executor.Executor: Running task 1.0 in stage 23.0 (TID 940)
17/06/09 20:10:56 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 32
17/06/09 20:10:56 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 951
17/06/09 20:10:56 INFO executor.Executor: Running task 2.0 in stage 23.0 (TID 951)
17/06/09 20:10:56 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 958
17/06/09 20:10:56 INFO executor.Executor: Running task 3.0 in stage 23.0 (TID 958)
</block>

<block lines="866-895" score="0.1285">
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_6_3 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_6_2 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_6_1 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_6_0 locally
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 40, boot = -340, init = 380, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 40, boot = -337, init = 376, finish = 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 42, boot = -343, init = 384, finish = 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 39, boot = -342, init = 381, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 41, boot = -343, init = 384, finish = 0
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000003_1012' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000003
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000004_1026' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000004
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000002_998' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000002
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000000_970' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000000
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000001_984' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000001
17/06/09 20:10:57 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000004_1026: Committed
17/06/09 20:10:57 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000002_998: Committed
17/06/09 20:10:57 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000003_1012: Committed
17/06/09 20:10:57 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000001_984: Committed
17/06/09 20:10:57 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000000_970: Committed
17/06/09 20:10:57 INFO executor.Executor: Finished task 3.0 in stage 24.0 (TID 1012). 2364 bytes result sent to driver
17/06/09 20:10:57 INFO executor.Executor: Finished task 4.0 in stage 24.0 (TID 1026). 2364 bytes result sent to driver
17/06/09 20:10:57 INFO executor.Executor: Finished task 1.0 in stage 24.0 (TID 984). 2364 bytes result sent to driver
17/06/09 20:10:57 INFO executor.Executor: Finished task 2.0 in stage 24.0 (TID 998). 2364 bytes result sent to driver
17/06/09 20:10:57 INFO executor.Executor: Finished task 0.0 in stage 24.0 (TID 970). 2364 bytes result sent to driver
17/06/09 20:10:57 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1125
</block>

<block lines="906-920" score="0.1298">
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_11_1 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_11_2 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_11_3 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_11_4 locally
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 39, boot = -397, init = 436, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 38, boot = -405, init = 443, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 40, boot = -400, init = 440, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 38, boot = -390, init = 428, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 41, boot = -398, init = 439, finish = 0
17/06/09 20:10:57 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000044_1129' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000044
</block>

<block lines="946-960" score="0.1279">
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_16_1 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_16_2 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_16_3 locally
17/06/09 20:10:57 INFO storage.BlockManager: Found block rdd_16_4 locally
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 37, boot = -87, init = 124, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 39, boot = -79, init = 118, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 39, boot = -86, init = 125, finish = 0
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 42, boot = -81, init = 122, finish = 1
17/06/09 20:10:57 INFO python.PythonRunner: Times: total = 41, boot = -83, init = 123, finish = 1
17/06/09 20:10:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000083_1138' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000083
</block>

<block lines="986-1005" score="0.1335">
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_21_1 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_21_0 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_21_2 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_21_3 locally
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 40, boot = -82, init = 122, finish = 0
17/06/09 20:10:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000124_1151' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000124
17/06/09 20:10:58 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000124_1151: Committed
17/06/09 20:10:58 INFO executor.Executor: Finished task 124.0 in stage 24.0 (TID 1151). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1155
17/06/09 20:10:58 INFO executor.Executor: Running task 160.0 in stage 24.0 (TID 1155)
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 39, boot = -102, init = 141, finish = 0
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 38, boot = -112, init = 150, finish = 0
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 40, boot = -107, init = 147, finish = 0
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 40, boot = -116, init = 156, finish = 0
17/06/09 20:10:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000120_1145' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000120
</block>

<block lines="1011-1020" score="0.1186">
17/06/09 20:10:58 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000121_1146: Committed
17/06/09 20:10:58 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000123_1150: Committed
17/06/09 20:10:58 INFO executor.Executor: Finished task 120.0 in stage 24.0 (TID 1145). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO executor.Executor: Finished task 121.0 in stage 24.0 (TID 1146). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO executor.Executor: Finished task 122.0 in stage 24.0 (TID 1147). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO executor.Executor: Finished task 123.0 in stage 24.0 (TID 1150). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1156
17/06/09 20:10:58 INFO executor.Executor: Running task 161.0 in stage 24.0 (TID 1156)
17/06/09 20:10:58 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1157
17/06/09 20:10:58 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1158
</block>

<block lines="1026-1035" score="0.1044">
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_26_3 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_26_4 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_26_1 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_26_2 locally
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 38, boot = -62, init = 100, finish = 0
17/06/09 20:10:58 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0024_m_000160_1155' to hdfs://10.10.34.11:9000/pjhe/test/1/_temporary/0/task_201706092018_0024_m_000160
17/06/09 20:10:58 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0024_m_000160_1155: Committed
17/06/09 20:10:58 INFO executor.Executor: Finished task 160.0 in stage 24.0 (TID 1155). 2364 bytes result sent to driver
17/06/09 20:10:58 INFO python.PythonRunner: Times: total = 38, boot = -45, init = 83, finish = 0
</block>

<block lines="1071-1085" score="0.1041">
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_30_2 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_30_1 locally
17/06/09 20:10:58 INFO storage.BlockManager: Found block rdd_30_0 locally
17/06/09 20:10:59 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:59 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:59 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:59 INFO python.PythonRunner: Times: total = 38, boot = -727, init = 765, finish = 0
17/06/09 20:10:59 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:59 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
17/06/09 20:10:59 INFO python.PythonRunner: Times: total = 41, boot = -707, init = 748, finish = 0
17/06/09 20:10:59 INFO python.PythonRunner: Times: total = 39, boot = -700, init = 739, finish = 0
17/06/09 20:10:59 INFO python.PythonRunner: Times: total = 39, boot = -690, init = 729, finish = 0
17/06/09 20:10:59 INFO python.PythonRunner: Times: total = 41, boot = -699, init = 740, finish = 0
17/06/09 20:10:59 INFO output.FileOutputCommitter: Saved output of task 'attempt_201706092018_0025_m_000004_1199' to hdfs://10.10.34.11:9000/pjhe/test/2/_temporary/0/task_201706092018_0025_m_000004
17/06/09 20:10:59 INFO mapred.SparkHadoopMapRedUtil: attempt_201706092018_0025_m_000004_1199: Committed
</block>

<block lines="1111-1130" score="0.1134">
17/06/09 20:11:07 INFO broadcast.TorrentBroadcast: Reading broadcast variable 37 took 14 ms
17/06/09 20:11:07 INFO storage.MemoryStore: Block broadcast_37 stored as values in memory (estimated size 8.8 KB, free 500.9 KB)
17/06/09 20:11:07 INFO spark.CacheManager: Partition rdd_42_0 not found, computing it
17/06/09 20:11:07 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:0+7292
17/06/09 20:11:07 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 36
17/06/09 20:11:07 INFO spark.CacheManager: Partition rdd_42_4 not found, computing it
17/06/09 20:11:07 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:29168+7292
17/06/09 20:11:07 INFO spark.CacheManager: Partition rdd_42_3 not found, computing it
17/06/09 20:11:07 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:21876+7292
17/06/09 20:11:07 INFO spark.CacheManager: Partition rdd_42_2 not found, computing it
17/06/09 20:11:07 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:14584+7292
17/06/09 20:11:07 INFO spark.CacheManager: Partition rdd_42_1 not found, computing it
17/06/09 20:11:07 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:7292+7292
17/06/09 20:11:07 INFO storage.MemoryStore: Block broadcast_36_piece0 stored as bytes in memory (estimated size 21.4 KB, free 522.3 KB)
17/06/09 20:11:07 INFO broadcast.TorrentBroadcast: Reading broadcast variable 36 took 11 ms
17/06/09 20:11:07 INFO storage.MemoryStore: Block broadcast_36 stored as values in memory (estimated size 281.6 KB, free 803.9 KB)
17/06/09 20:11:07 INFO python.PythonRunner: Times: total = 47, boot = -8758, init = 8800, finish = 5
17/06/09 20:11:07 INFO storage.MemoryStore: Block rdd_42_0 stored as bytes in memory (estimated size 913.0 B, free 804.8 KB)
17/06/09 20:11:07 INFO python.PythonRunner: Times: total = 55, boot = -8764, init = 8814, finish = 5
17/06/09 20:11:07 INFO python.PythonRunner: Times: total = 55, boot = -8768, init = 8818, finish = 5
</block>

<block lines="1316-1325" score="0.1270">
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:196884+7292
17/06/09 20:11:08 INFO executor.Executor: Finished task 23.0 in stage 26.0 (TID 1223). 2703 bytes result sent to driver
17/06/09 20:11:08 INFO executor.Executor: Finished task 24.0 in stage 26.0 (TID 1224). 2703 bytes result sent to driver
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1228
17/06/09 20:11:08 INFO executor.Executor: Running task 28.0 in stage 26.0 (TID 1228)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1229
17/06/09 20:11:08 INFO executor.Executor: Running task 29.0 in stage 26.0 (TID 1229)
17/06/09 20:11:08 INFO spark.CacheManager: Partition rdd_42_28 not found, computing it
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:204176+7292
17/06/09 20:11:08 INFO spark.CacheManager: Partition rdd_42_29 not found, computing it
</block>

<block lines="1346-1355" score="0.0763">
17/06/09 20:11:08 INFO executor.Executor: Finished task 27.0 in stage 26.0 (TID 1227). 2701 bytes result sent to driver
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1231
17/06/09 20:11:08 INFO executor.Executor: Running task 31.0 in stage 26.0 (TID 1231)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1232
17/06/09 20:11:08 INFO executor.Executor: Running task 32.0 in stage 26.0 (TID 1232)
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 42, boot = 22, init = 19, finish = 1
17/06/09 20:11:08 INFO spark.CacheManager: Partition rdd_42_31 not found, computing it
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:226052+7292
17/06/09 20:11:08 INFO spark.CacheManager: Partition rdd_42_32 not found, computing it
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:233344+7292
</block>

<block lines="1381-1390" score="0.0722">
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:255220+7292
17/06/09 20:11:08 INFO storage.MemoryStore: Block rdd_42_33 stored as bytes in memory (estimated size 869.0 B, free 833.7 KB)
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 42, boot = 24, init = 18, finish = 0
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 38, boot = 23, init = 15, finish = 0
17/06/09 20:11:08 INFO executor.Executor: Finished task 32.0 in stage 26.0 (TID 1232). 2703 bytes result sent to driver
17/06/09 20:11:08 INFO executor.Executor: Finished task 31.0 in stage 26.0 (TID 1231). 2703 bytes result sent to driver
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1236
17/06/09 20:11:08 INFO executor.Executor: Running task 36.0 in stage 26.0 (TID 1236)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1237
17/06/09 20:11:08 INFO executor.Executor: Running task 37.0 in stage 26.0 (TID 1237)
</block>

<block lines="1406-1415" score="0.0677">
17/06/09 20:11:08 INFO spark.CacheManager: Partition rdd_42_39 not found, computing it
17/06/09 20:11:08 INFO rdd.HadoopRDD: Input split: hdfs://10.10.34.11:9000/pjhe/logs/2kSOSP.log:284388+7303
17/06/09 20:11:08 INFO storage.MemoryStore: Block rdd_42_35 stored as bytes in memory (estimated size 917.0 B, free 834.6 KB)
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 53, boot = 8, init = 40, finish = 5
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 53, boot = 12, init = 36, finish = 5
17/06/09 20:11:08 INFO storage.MemoryStore: Block rdd_42_37 stored as bytes in memory (estimated size 939.0 B, free 835.5 KB)
17/06/09 20:11:08 INFO storage.MemoryStore: Block rdd_42_36 stored as bytes in memory (estimated size 912.0 B, free 836.4 KB)
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 41, boot = 9, init = 31, finish = 1
17/06/09 20:11:08 INFO executor.Executor: Finished task 35.0 in stage 26.0 (TID 1235). 2705 bytes result sent to driver
17/06/09 20:11:08 INFO python.PythonRunner: Times: total = 48, boot = 17, init = 25, finish = 6
</block>

<block lines="1426-1435" score="0.0641">
17/06/09 20:11:08 INFO executor.Executor: Finished task 39.0 in stage 26.0 (TID 1239). 2703 bytes result sent to driver
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1240
17/06/09 20:11:08 INFO executor.Executor: Running task 0.0 in stage 27.0 (TID 1240)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1241
17/06/09 20:11:08 INFO executor.Executor: Running task 1.0 in stage 27.0 (TID 1241)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1242
17/06/09 20:11:08 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 38
17/06/09 20:11:08 INFO executor.Executor: Running task 2.0 in stage 27.0 (TID 1242)
17/06/09 20:11:08 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1243
17/06/09 20:11:08 INFO executor.Executor: Running task 3.0 in stage 27.0 (TID 1243)
</block>

<block lines="1731-1740" score="0.0811">
17/06/09 20:11:10 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1300
17/06/09 20:11:10 INFO executor.Executor: Running task 20.0 in stage 28.0 (TID 1300)
17/06/09 20:11:10 INFO python.PythonRunner: Times: total = 40, boot = 15, init = 25, finish = 0
17/06/09 20:11:10 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1301
17/06/09 20:11:10 INFO python.PythonRunner: Times: total = 42, boot = 16, init = 25, finish = 1
17/06/09 20:11:10 INFO executor.Executor: Running task 21.0 in stage 28.0 (TID 1301)
17/06/09 20:11:10 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1302
17/06/09 20:11:10 INFO executor.Executor: Running task 22.0 in stage 28.0 (TID 1302)
17/06/09 20:11:10 INFO storage.BlockManager: Found block rdd_42_20 locally
17/06/09 20:11:10 INFO executor.Executor: Finished task 18.0 in stage 28.0 (TID 1298). 2087 bytes result sent to driver
</block>
