Query (id=684369c252187019:899cc5d500000000): DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. Summary: Session ID: ca4589f75c58abec:e0b7aecbb7f095b4 Session Type: BEESWAX Start Time: 2017-06-13 14:23:44.448925000 End Time: 2017-06-13 14:23:46.221584000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT DEBUG (build a2281508ff64b9ef3ea730cfd98f6a1c7a497766) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:32802 Default Db: tpch Sql Statement: select count(t1.ps_suppkey) from partsupp t1 LEFT JOIN ( SELECT COALESCE(t2.o_custkey, -1) AS coalesce_col FROM orders t2 LEFT JOIN partsupp t3 ON (t2.o_shippriority) = (t3.ps_partkey) ) sq ON ((t1.ps_suppkey) = (sq.coalesce_col)) Coordinator: mikeb-desktop:22000 Query Options (non default): Plan: ---------------- Per-Host Resource Reservation: Memory=272.00MB Per-Host Resource Estimates: Memory=365.43MB F04:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 10:AGGREGATE [FINALIZE] | output: count:merge(t1.ps_suppkey) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | 09:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | F03:PLAN FRAGMENT [HASH(if(TupleIsNull(), NULL, coalesce(t2.o_custkey, -1)))] hosts=1 instances=1 05:AGGREGATE | output: count(t1.ps_suppkey) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | 04:HASH JOIN [RIGHT OUTER JOIN, PARTITIONED] | hash predicates: if(TupleIsNull(), NULL, coalesce(t2.o_custkey, -1)) = (t1.ps_suppkey) | runtime filters: RF000 <- (t1.ps_suppkey) | mem-estimate=6.71MB mem-reservation=136.00MB | tuple-ids=1N,2N,0 row-size=28B cardinality=800000 | |--08:EXCHANGE [HASH((t1.ps_suppkey))] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=0 row-size=8B cardinality=800000 | | | F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | 00:SCAN HDFS [tpch.partsupp t1, RANDOM] | partitions=1/1 files=1 size=112.71MB | stats-rows=800000 extrapolated-rows=disabled | table stats: rows=800000 size=unavailable | column stats: all | mem-estimate=176.00MB mem-reservation=0B | tuple-ids=0 row-size=8B cardinality=800000 | 07:EXCHANGE [HASH(if(TupleIsNull(), NULL, coalesce(t2.o_custkey, -1)))] | mem-estimate=0B mem-reservation=0B | tuple-ids=1,2N row-size=20B cardinality=1500000 | F00:PLAN FRAGMENT [RANDOM] hosts=2 instances=2 03:HASH JOIN [LEFT OUTER JOIN, BROADCAST] | hash predicates: (t2.o_shippriority) = (t3.ps_partkey) | mem-estimate=6.71MB mem-reservation=136.00MB | tuple-ids=1,2N row-size=20B cardinality=1500000 | |--06:EXCHANGE [BROADCAST] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=2 row-size=8B cardinality=800000 | | | F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | 02:SCAN HDFS [tpch.partsupp t3, RANDOM] | partitions=1/1 files=1 size=112.71MB | stats-rows=800000 extrapolated-rows=disabled | table stats: rows=800000 size=unavailable | column stats: all | mem-estimate=176.00MB mem-reservation=0B | tuple-ids=2 row-size=8B cardinality=800000 | 01:SCAN HDFS [tpch.orders t2, RANDOM] partitions=1/1 files=1 size=162.56MB runtime filters: RF000 -> coalesce(t2.o_custkey, -1) stats-rows=1500000 extrapolated-rows=disabled table stats: rows=1500000 size=unavailable column stats: all mem-estimate=176.00MB mem-reservation=0B tuple-ids=1 row-size=12B cardinality=1500000 ---------------- Estimated Per-Host Mem: 383178754 Per-Host Memory Reservation: 285212672 Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------ 10:AGGREGATE 1 198.204us 198.204us 1 1 24.00 KB 10.00 MB FINALIZE 09:EXCHANGE 1 93.555us 93.555us 2 1 0 0 UNPARTITIONED 05:AGGREGATE 2 3.066ms 3.717ms 2 1 8.04 MB 10.00 MB 04:HASH JOIN 2 140.679ms 236.316ms 800.00K 800.00K 114.06 MB 6.71 MB RIGHT OUTER JOIN, PARTITIONED |--08:EXCHANGE 2 51.184ms 53.408ms 800.00K 800.00K 0 0 HASH((t1.ps_suppkey)) | 00:SCAN HDFS 1 9.601ms 9.601ms 800.00K 800.00K 33.02 MB 176.00 MB tpch.partsupp t1 07:EXCHANGE 2 88.551ms 177.099ms 1.50M 1.50M 0 0 HASH(if(TupleIsNull(), NULL... 03:HASH JOIN 2 216.025ms 241.128ms 1.50M 1.50M 178.05 MB 6.71 MB LEFT OUTER JOIN, BROADCAST |--06:EXCHANGE 2 93.506ms 105.640ms 800.00K 800.00K 0 0 BROADCAST | 02:SCAN HDFS 1 112.021ms 112.021ms 800.00K 800.00K 32.63 MB 176.00 MB tpch.partsupp t3 01:SCAN HDFS 2 50.246ms 86.962ms 1.50M 1.50M 33.25 MB 176.00 MB tpch.orders t2 Errors: Planner Timeline: 9.498ms - Analysis finished: 2.763ms (2.763ms) - Equivalence classes computed: 2.921ms (157.529us) - Single node plan created: 5.984ms (3.063ms) - Runtime filters computed: 6.189ms (204.944us) - Distributed plan created: 6.901ms (711.833us) - Planning finished: 9.498ms (2.597ms) Query Timeline: 1s776ms - Query submitted: 75.888us (75.888us) - Planning finished: 10.709ms (10.633ms) - Submit for admission: 11.030ms (320.706us) - Completed admission: 11.067ms (37.606us) - Ready to start on 2 backends: 11.425ms (357.375us) - All 2 execution backends (7 fragment instances) started: 12.620ms (1.195ms) - First dynamic filter received: 682.967ms (670.346ms) - Rows available: 1s750ms (1s067ms) - First row fetched: 1s768ms (18.202ms) - Unregister query: 1s772ms (3.892ms) - ComputeScanRangeAssignmentTimer: 91.404us ImpalaServer: - ClientFetchWaitTimer: 19.657ms - RowMaterializationTimer: 2.417ms Execution Profile 684369c252187019:899cc5d500000000:(Total: 1s741ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 1 REMOTE false 2 (2) N/A N/A true Backend startup latencies: Count: 2, min / max: 1ms / 1ms, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 1 REMOTE false 0 (2) 682.968ms 684.083ms false Per Node Peak Memory Usage: mikeb-desktop:22000(327.51 MB) mikeb-desktop:22001(319.91 MB) - FiltersReceived: 2 (2) - FinalizationTimer: 0.000ns Averaged Fragment F04:(Total: 1s756ms, non-child: 18.771ms, % non-child: 1.07%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1s756ms max:1s756ms mean: 1s756ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.02 KB (32784) - PerHostPeakMemUsage: 327.51 MB (343421477) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s718ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 1s736ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 3 (3) Fragment Instance Lifecycle Timings: - ExecTime: 18.309ms - ExecTreeExecTime: 3.195us - OpenTime: 1s718ms - ExecTreeOpenTime: 1s718ms - PrepareTime: 19.496ms - ExecTreePrepareTime: 101.890us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 154 (154) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 256.00 MB (268435456) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 19.030ms, non-child: 19.030ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 18.500ms AGGREGATION_NODE (id=10):(Total: 1s718ms, non-child: 198.204us, % non-child: 0.01%) - BuildTime: 5.159us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=9):(Total: 1s718ms, non-child: 1s718ms, % non-child: 100.00%) - BytesReceived: 32.00 B (32) - ConvertRowBatchTime: 2.099us - DeserializeRowBatchTimer: 26.936us - FirstBatchArrivalWaitTime: 1s718ms - PeakMemoryUsage: 0 - RowsReturned: 2 (2) - RowsReturnedRate: 1.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Coordinator Fragment F04: Instance 684369c252187019:899cc5d500000000 (host=mikeb-desktop:22000):(Total: 1s756ms, non-child: 18.771ms, % non-child: 1.07%) MemoryUsage(500.000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.02 KB (32784) - PerHostPeakMemUsage: 327.51 MB (343421477) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s718ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1 (1) - TotalThreadsTotalWallClockTime: 1s736ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 3 (3) Fragment Instance Lifecycle Timings: - ExecTime: 18.309ms - ExecTreeExecTime: 3.195us - OpenTime: 1s718ms - ExecTreeOpenTime: 1s718ms - PrepareTime: 19.496ms - ExecTreePrepareTime: 101.890us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 154 (154) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 256.00 MB (268435456) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 19.030ms, non-child: 19.030ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 18.500ms AGGREGATION_NODE (id=10):(Total: 1s718ms, non-child: 198.204us, % non-child: 0.01%) - BuildTime: 5.159us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=9):(Total: 1s718ms, non-child: 93.555us, % non-child: 0.01%) BytesReceived(500.000ms): 0, 0, 0, 0 - BytesReceived: 32.00 B (32) - ConvertRowBatchTime: 2.099us - DeserializeRowBatchTimer: 26.936us - FirstBatchArrivalWaitTime: 1s718ms - PeakMemoryUsage: 0 - RowsReturned: 2 (2) - RowsReturnedRate: 1.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F03:(Total: 1s737ms, non-child: 7.067ms, % non-child: 0.41%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1s738ms max:1s738ms mean: 1s738ms stddev:132.225us execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 2 - AverageThreadTokens: 1.25 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 115.15 MB (120742493) - PerHostPeakMemUsage: 323.71 MB (339436721) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s591ms - TotalNetworkSendTime: 268.980us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 207 (207) - TotalThreadsTotalWallClockTime: 2s226ms - TotalThreadsSysTime: 6.000ms - TotalThreadsUserTime: 622.000ms - TotalThreadsVoluntaryContextSwitches: 1.05K (1046) Fragment Instance Lifecycle Timings: - ExecTime: 439.117us - ExecTreeExecTime: 3.627us - OpenTime: 1s713ms - ExecTreeOpenTime: 1s528ms - PrepareTime: 23.613ms - ExecTreePrepareTime: 294.890us DataStreamSender (dst_id=9):(Total: 68.447us, non-child: 68.447us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 656.44 KB/sec - OverallThroughput: 239.14 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 14.510us - TransmitDataRPCTime: 33.710us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 202.184ms, non-child: 202.184ms, % non-child: 100.00%) - CodegenTime: 389.496us - CompileTime: 17.883ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 61 (61) - NumInstructions: 2.51K (2510) - OptimizationTime: 166.786ms - PeakMemoryUsage: 1.23 MB (1285120) - PrepareTime: 17.092ms AGGREGATION_NODE (id=5):(Total: 1s528ms, non-child: 3.066ms, % non-child: 0.20%) - BuildTime: 2.661ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 4.04 MB (4232464) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s525ms, non-child: 140.679ms, % non-child: 9.22%) - BuildRows: 400.00K (400000) - BuildTime: 172.097ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 110.06 MB (115410632) - ProbeRows: 750.00K (750000) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 118.467ms - RowsReturned: 400.00K (400000) - RowsReturnedRate: 262.25 K/sec Hash Join Builder (join_node_id=4): - BuildRowsPartitionTime: 119.640ms - BuildRowsPartitioned: 400.00K (400000) - GetNewBlockTime: 100.280us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 51.602ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 110.02 MB (115364992) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 339.385ms, non-child: 339.385ms, % non-child: 100.00%) - BytesReceived: 3.06 MB (3208099) - ConvertRowBatchTime: 48.478ms - DeserializeRowBatchTimer: 11.385ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 400.00K (400000) - RowsReturnedRate: 1.18 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 1s385ms, non-child: 1s385ms, % non-child: 100.00%) - BytesReceived: 9.23 MB (9680715) - ConvertRowBatchTime: 78.187ms - DeserializeRowBatchTimer: 35.093ms - FirstBatchArrivalWaitTime: 984.785ms - PeakMemoryUsage: 0 - RowsReturned: 750.00K (750000) - RowsReturnedRate: 588.37 K/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 153 (153) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 248.00 MB (260046848) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Fragment F03: Instance 684369c252187019:899cc5d500000005 (host=mikeb-desktop:22001):(Total: 1s737ms, non-child: 7.034ms, % non-child: 0.40%) MemoryUsage(500.000ms): 1.54 MB, 10.13 MB, 108.68 MB, 108.35 MB ThreadUsage(500.000ms): 1, 2, 1, 1 - AverageThreadTokens: 1.25 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 115.17 MB (120769461) - PerHostPeakMemUsage: 319.91 MB (335451965) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s394ms - TotalNetworkSendTime: 293.392us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 270 (270) - TotalThreadsTotalWallClockTime: 2s208ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 812.000ms - TotalThreadsVoluntaryContextSwitches: 1.91K (1907) Fragment Instance Lifecycle Timings: - ExecTime: 470.744us - ExecTreeExecTime: 4.043us - OpenTime: 1s710ms - ExecTreeOpenTime: 1s514ms - PrepareTime: 26.557ms - ExecTreePrepareTime: 313.179us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 153 (153) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 248.00 MB (260046848) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=9):(Total: 83.038us, non-child: 83.038us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 300.57 KB/sec - OverallThroughput: 188.17 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 9.433us - TransmitDataRPCTime: 51.985us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 216.352ms, non-child: 216.352ms, % non-child: 100.00%) - CodegenTime: 380.162us - CompileTime: 18.318ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 61 (61) - NumInstructions: 2.51K (2510) - OptimizationTime: 177.615ms - PeakMemoryUsage: 1.23 MB (1285120) - PrepareTime: 19.946ms AGGREGATION_NODE (id=5):(Total: 1s514ms, non-child: 3.717ms, % non-child: 0.25%) ExecOption: Codegen Enabled - BuildTime: 3.339ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 8.04 MB (8432160) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s510ms, non-child: 236.316ms, % non-child: 15.64%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 399.12K (399120) - BuildTime: 162.662ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 106.07 MB (111223952) - ProbeRows: 1.50M (1500000) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 192.797ms - RowsReturned: 399.12K (399120) - RowsReturnedRate: 264.20 K/sec Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 110.728ms - BuildRowsPartitioned: 399.12K (399120) - GetNewBlockTime: 104.934us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 51.009ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 106.02 MB (111170688) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 333.694ms, non-child: 48.960ms, % non-child: 14.67%) BytesReceived(500.000ms): 114.00 KB, 2.69 MB, 3.05 MB, 3.05 MB - BytesReceived: 3.05 MB (3201640) - ConvertRowBatchTime: 46.101ms - DeserializeRowBatchTimer: 10.870ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 399.12K (399120) - RowsReturnedRate: 1.20 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 1s274ms, non-child: 177.099ms, % non-child: 13.89%) BytesReceived(500.000ms): 0, 0, 9.47 MB, 17.11 MB - BytesReceived: 18.46 MB (19361431) - ConvertRowBatchTime: 156.373ms - DeserializeRowBatchTimer: 70.186ms - FirstBatchArrivalWaitTime: 474.142ms - PeakMemoryUsage: 0 - RowsReturned: 1.50M (1500000) - RowsReturnedRate: 1.18 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance 684369c252187019:899cc5d500000006 (host=mikeb-desktop:22000):(Total: 1s737ms, non-child: 7.099ms, % non-child: 0.41%) MemoryUsage(500.000ms): 1.57 MB, 10.13 MB, 115.10 MB, 115.10 MB ThreadUsage(500.000ms): 1, 2, 1, 1 - AverageThreadTokens: 1.25 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 115.12 MB (120715525) - PerHostPeakMemUsage: 327.51 MB (343421477) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s788ms - TotalNetworkSendTime: 244.568us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 144 (144) - TotalThreadsTotalWallClockTime: 2s244ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 432.000ms - TotalThreadsVoluntaryContextSwitches: 186 (186) Fragment Instance Lifecycle Timings: - ExecTime: 407.491us - ExecTreeExecTime: 3.210us - OpenTime: 1s716ms - ExecTreeOpenTime: 1s542ms - PrepareTime: 20.669ms - ExecTreePrepareTime: 276.601us DataStreamSender (dst_id=9):(Total: 53.857us, non-child: 53.857us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 1012.31 KB/sec - OverallThroughput: 290.12 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 19.588us - TransmitDataRPCTime: 15.435us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 188.017ms, non-child: 188.017ms, % non-child: 100.00%) - CodegenTime: 398.831us - CompileTime: 17.448ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 61 (61) - NumInstructions: 2.51K (2510) - OptimizationTime: 155.956ms - PeakMemoryUsage: 1.23 MB (1285120) - PrepareTime: 14.239ms AGGREGATION_NODE (id=5):(Total: 1s542ms, non-child: 2.416ms, % non-child: 0.16%) ExecOption: Codegen Enabled - BuildTime: 1.984ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 32.00 KB (32768) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s540ms, non-child: 45.043ms, % non-child: 2.92%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 400.88K (400880) - BuildTime: 181.532ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 114.06 MB (119597312) - ProbeRows: 0 (0) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 44.137ms - RowsReturned: 400.88K (400880) - RowsReturnedRate: 260.29 K/sec Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 128.551ms - BuildRowsPartitioned: 400.88K (400880) - GetNewBlockTime: 95.627us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 52.195ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 114.02 MB (119559296) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 345.076ms, non-child: 53.408ms, % non-child: 15.48%) BytesReceived(500.000ms): 128.21 KB, 2.69 MB, 3.07 MB, 3.07 MB - BytesReceived: 3.07 MB (3214558) - ConvertRowBatchTime: 50.856ms - DeserializeRowBatchTimer: 11.900ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 400.88K (400880) - RowsReturnedRate: 1.16 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 1s495ms, non-child: 3.302us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0 - BytesReceived: 0 - ConvertRowBatchTime: 820.000ns - DeserializeRowBatchTimer: 0.000ns - FirstBatchArrivalWaitTime: 1s495ms - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1s313ms, non-child: 259.558ms, % non-child: 19.76%) split sizes: min: 34.56 MB, max: 128.00 MB, avg: 81.28 MB, stddev: 46.72 MB completion times: min:937.049ms max:1s691ms mean: 1s314ms stddev:377.289ms execution rates: min:36.88 MB/sec max:75.67 MB/sec mean:56.27 MB/sec stddev:19.39 MB/sec num instances: 2 - AverageThreadTokens: 1.38 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 212.08 MB (222380832) - PerHostPeakMemUsage: 323.71 MB (339436721) - RowsProduced: 750.00K (750000) - TotalNetworkReceiveTime: 3.875ms - TotalNetworkSendTime: 10.335ms - TotalStorageWaitTime: 2.554ms - TotalThreadsInvoluntaryContextSwitches: 421 (421) - TotalThreadsTotalWallClockTime: 2s132ms - TotalThreadsSysTime: 6.000ms - TotalThreadsUserTime: 1s674ms - TotalThreadsVoluntaryContextSwitches: 643 (643) Fragment Instance Lifecycle Timings: - ExecTime: 618.881ms - ExecTreeExecTime: 45.486ms - OpenTime: 659.079ms - ExecTreeOpenTime: 313.974ms - PrepareTime: 35.400ms - ExecTreePrepareTime: 447.708us DataStreamSender (dst_id=7):(Total: 331.768ms, non-child: 331.768ms, % non-child: 100.00%) - BytesSent: 9.23 MB (9680715) - NetworkThroughput(*): 127.74 MB/sec - OverallThroughput: 28.49 MB/sec - PeakMemoryUsage: 35.25 KB (36096) - RowsReturned: 750.00K (750000) - SerializeBatchTime: 261.171ms - TransmitDataRPCTime: 70.427ms - UncompressedRowBatchSize: 15.03 MB (15758072) CodeGen:(Total: 362.319ms, non-child: 362.319ms, % non-child: 100.00%) - CodegenTime: 1.445ms - CompileTime: 29.830ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 314.147ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 17.818ms HASH_JOIN_NODE (id=3):(Total: 359.775ms, non-child: 216.025ms, % non-child: 60.04%) - BuildRows: 800.00K (800000) - BuildTime: 175.130ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 750.00K (750000) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 33.645ms - RowsReturned: 750.00K (750000) - RowsReturnedRate: 2.17 M/sec Hash Join Builder (join_node_id=3): - BuildRowsPartitionTime: 73.633ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 86.113us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 100.896ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 93.506ms, non-child: 93.506ms, % non-child: 100.00%) - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 86.857ms - DeserializeRowBatchTimer: 25.239ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 8.70 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 50.246ms, non-child: 50.246ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.33 - BytesRead: 81.31 MB (85258848) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.31 MB (85258848) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.31 MB (85258848) - CachedFileHandlesHitCount: 11 (11) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 32.94 MB (34536960) - PerReadThreadRawHdfsThroughput: 3.13 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.999ms - RowBatchQueuePutWaitTime: 108.190ms - RowsRead: 750.00K (750000) - RowsReturned: 750.00K (750000) - RowsReturnedRate: 45.49 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 77 (77) - ScannerThreadsTotalWallClockTime: 583.924ms - DelimiterParseTime: 413.959ms - MaterializeTupleTime(*): 53.666ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 472.000ms - ScannerThreadsVoluntaryContextSwitches: 507 (507) - TotalRawHdfsReadTime(*): 23.188ms - TotalReadThroughput: 32.00 MB/sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Fragment F00: Instance 684369c252187019:899cc5d500000003 (host=mikeb-desktop:22000):(Total: 1s690ms, non-child: 422.657ms, % non-child: 25.00%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB Filter 0 arrival: 672ms MemoryUsage(500.000ms): 2.17 MB, 72.49 MB, 204.39 MB, 188.40 MB ThreadUsage(500.000ms): 1, 2, 2, 2 - AverageThreadTokens: 1.75 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 212.40 MB (222714144) - PerHostPeakMemUsage: 327.51 MB (343421477) - RowsProduced: 1.18M (1181503) - TotalNetworkReceiveTime: 3.096ms - TotalNetworkSendTime: 17.669ms - TotalStorageWaitTime: 2.644ms - TotalThreadsInvoluntaryContextSwitches: 554 (554) - TotalThreadsTotalWallClockTime: 2s861ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 2s344ms - TotalThreadsVoluntaryContextSwitches: 1.18K (1182) Fragment Instance Lifecycle Timings: - ExecTime: 997.892ms - ExecTreeExecTime: 72.855ms - OpenTime: 642.447ms - ExecTreeOpenTime: 262.899ms - PrepareTime: 50.095ms - ExecTreePrepareTime: 565.335us DataStreamSender (dst_id=7):(Total: 530.825ms, non-child: 530.825ms, % non-child: 100.00%) - BytesSent: 14.54 MB (15242487) - NetworkThroughput(*): 133.92 MB/sec - OverallThroughput: 27.38 MB/sec - PeakMemoryUsage: 35.25 KB (36096) - RowsReturned: 1.18M (1181503) - SerializeBatchTime: 422.026ms - TransmitDataRPCTime: 108.542ms - UncompressedRowBatchSize: 23.67 MB (24824275) CodeGen:(Total: 400.997ms, non-child: 400.997ms, % non-child: 100.00%) - CodegenTime: 1.745ms - CompileTime: 34.284ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 343.994ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 22.077ms HASH_JOIN_NODE (id=3):(Total: 336.027ms, non-child: 241.128ms, % non-child: 71.76%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 800.00K (800000) - BuildTime: 176.638ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 1.18M (1181503) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 55.365ms - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 3.52 M/sec Hash Join Builder (join_node_id=3): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 64.998ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 91.196us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 110.752ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 81.371ms, non-child: 81.371ms, % non-child: 100.00%) BytesReceived(500.000ms): 228.48 KB, 4.59 MB, 4.59 MB, 4.59 MB - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 75.676ms - DeserializeRowBatchTimer: 25.183ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 9.83 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 13.530ms, non-child: 13.530ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0, 0, 72.00 MB, 128.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.67 - BytesRead: 128.06 MB (134283264) - BytesReadDataNodeCache: 0 - BytesReadLocal: 128.06 MB (134283264) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 128.06 MB (134283264) - CachedFileHandlesHitCount: 17 (17) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 2 (2) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.25 MB (34870272) - PerReadThreadRawHdfsThroughput: 3.91 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.578ms - RowBatchQueuePutWaitTime: 216.380ms - RowsRead: 1.18M (1181503) - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 87.32 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 128 (128) - ScannerThreadsTotalWallClockTime: 961.425ms - DelimiterParseTime: 649.215ms - MaterializeTupleTime(*): 84.143ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 744.000ms - ScannerThreadsVoluntaryContextSwitches: 1.01K (1013) - TotalRawHdfsReadTime(*): 31.955ms - TotalReadThroughput: 64.00 MB/sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Instance 684369c252187019:899cc5d500000002 (host=mikeb-desktop:22001):(Total: 936.337ms, non-child: 96.460ms, % non-child: 10.30%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB Filter 0 arrival: 673ms MemoryUsage(500.000ms): 2.17 MB, 178.10 MB ThreadUsage(500.000ms): 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 211.76 MB (222047520) - PerHostPeakMemUsage: 319.91 MB (335451965) - RowsProduced: 318.50K (318497) - TotalNetworkReceiveTime: 4.654ms - TotalNetworkSendTime: 3.001ms - TotalStorageWaitTime: 2.464ms - TotalThreadsInvoluntaryContextSwitches: 288 (288) - TotalThreadsTotalWallClockTime: 1s403ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 1s004ms - TotalThreadsVoluntaryContextSwitches: 105 (105) Fragment Instance Lifecycle Timings: - ExecTime: 239.869ms - ExecTreeExecTime: 18.117ms - OpenTime: 675.710ms - ExecTreeOpenTime: 365.048ms - PrepareTime: 20.704ms - ExecTreePrepareTime: 330.081us DataStreamSender (dst_id=7):(Total: 132.711ms, non-child: 132.711ms, % non-child: 100.00%) - BytesSent: 3.93 MB (4118944) - NetworkThroughput(*): 121.57 MB/sec - OverallThroughput: 29.60 MB/sec - PeakMemoryUsage: 35.25 KB (36096) - RowsReturned: 318.50K (318497) - SerializeBatchTime: 100.316ms - TransmitDataRPCTime: 32.312ms - UncompressedRowBatchSize: 6.38 MB (6691869) CodeGen:(Total: 323.642ms, non-child: 323.642ms, % non-child: 100.00%) - CodegenTime: 1.145ms - CompileTime: 25.377ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 284.300ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 13.558ms HASH_JOIN_NODE (id=3):(Total: 383.523ms, non-child: 190.923ms, % non-child: 49.78%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 800.00K (800000) - BuildTime: 173.621ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 318.50K (318497) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 11.926ms - RowsReturned: 318.50K (318497) - RowsReturnedRate: 830.45 K/sec Hash Join Builder (join_node_id=3): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 82.268ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 81.030us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 91.040ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 105.640ms, non-child: 105.640ms, % non-child: 100.00%) BytesReceived(500.000ms): 228.48 KB, 4.59 MB - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 98.039ms - DeserializeRowBatchTimer: 25.296ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 7.57 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 86.962ms, non-child: 86.962ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 80ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0, 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 34.56 MB (36234433) - BytesReadDataNodeCache: 0 - BytesReadLocal: 34.56 MB (36234433) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 34.56 MB (36234433) - CachedFileHandlesHitCount: 5 (5) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 32.62 MB (34203648) - PerReadThreadRawHdfsThroughput: 2.34 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 3.420ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 318.50K (318497) - RowsReturned: 318.50K (318497) - RowsReturnedRate: 3.66 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 26 (26) - ScannerThreadsTotalWallClockTime: 206.423ms - DelimiterParseTime: 178.704ms - MaterializeTupleTime(*): 23.188ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 200.000ms - ScannerThreadsVoluntaryContextSwitches: 1 (1) - TotalRawHdfsReadTime(*): 14.420ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Averaged Fragment F01:(Total: 518.243ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 112.71 MB, max: 112.71 MB, avg: 112.71 MB, stddev: 0 completion times: min:519.275ms max:519.275ms mean: 519.275ms stddev:0.000ns execution rates: min:217.05 MB/sec max:217.05 MB/sec mean:217.05 MB/sec stddev:0.39 B/sec num instances: 1 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.68 MB (34269764) - PerHostPeakMemUsage: 78.05 MB (81839721) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 86.319ms - TotalStorageWaitTime: 2.058ms - TotalThreadsInvoluntaryContextSwitches: 484 (484) - TotalThreadsTotalWallClockTime: 921.521ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 624.000ms - TotalThreadsVoluntaryContextSwitches: 493 (493) Fragment Instance Lifecycle Timings: - ExecTime: 446.635ms - ExecTreeExecTime: 112.552ms - OpenTime: 53.832ms - ExecTreeOpenTime: 76.732us - PrepareTime: 17.732ms - ExecTreePrepareTime: 86.189us DataStreamSender (dst_id=6):(Total: 400.640ms, non-child: 400.640ms, % non-child: 100.00%) - BytesSent: 9.18 MB (9621540) - NetworkThroughput(*): 57.10 MB/sec - OverallThroughput: 22.90 MB/sec - PeakMemoryUsage: 28.44 KB (29120) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 239.776ms - TransmitDataRPCTime: 160.704ms - UncompressedRowBatchSize: 19.84 MB (20806256) CodeGen:(Total: 70.649ms, non-child: 70.649ms, % non-child: 100.00%) - CodegenTime: 1.050ms - CompileTime: 5.150ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 47.596ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 17.370ms HDFS_SCAN_NODE (id=2):(Total: 112.021ms, non-child: 112.021ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 32.63 MB (34220032) - PerReadThreadRawHdfsThroughput: 3.86 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 103.601ms - RowBatchQueuePutWaitTime: 6.183ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 7.14 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 256 (256) - ScannerThreadsTotalWallClockTime: 421.063ms - DelimiterParseTime: 304.580ms - MaterializeTupleTime(*): 38.983ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 348.000ms - ScannerThreadsVoluntaryContextSwitches: 20 (20) - TotalRawHdfsReadTime(*): 28.517ms - TotalReadThroughput: 48.00 MB/sec Fragment F01: Instance 684369c252187019:899cc5d500000001 (host=mikeb-desktop:22000):(Total: 518.243ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB MemoryUsage(500.000ms): 24.04 MB ThreadUsage(500.000ms): 2 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.68 MB (34269764) - PerHostPeakMemUsage: 78.05 MB (81839721) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 86.319ms - TotalStorageWaitTime: 2.058ms - TotalThreadsInvoluntaryContextSwitches: 484 (484) - TotalThreadsTotalWallClockTime: 921.521ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 624.000ms - TotalThreadsVoluntaryContextSwitches: 493 (493) Fragment Instance Lifecycle Timings: - ExecTime: 446.635ms - ExecTreeExecTime: 112.552ms - OpenTime: 53.832ms - ExecTreeOpenTime: 76.732us - PrepareTime: 17.732ms - ExecTreePrepareTime: 86.189us DataStreamSender (dst_id=6):(Total: 400.640ms, non-child: 400.640ms, % non-child: 100.00%) - BytesSent: 9.18 MB (9621540) - NetworkThroughput(*): 57.10 MB/sec - OverallThroughput: 22.90 MB/sec - PeakMemoryUsage: 28.44 KB (29120) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 239.776ms - TransmitDataRPCTime: 160.704ms - UncompressedRowBatchSize: 19.84 MB (20806256) CodeGen:(Total: 70.649ms, non-child: 70.649ms, % non-child: 100.00%) - CodegenTime: 1.050ms - CompileTime: 5.150ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 47.596ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 17.370ms HDFS_SCAN_NODE (id=2):(Total: 112.021ms, non-child: 112.021ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 24.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 32.63 MB (34220032) - PerReadThreadRawHdfsThroughput: 3.86 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 103.601ms - RowBatchQueuePutWaitTime: 6.183ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 7.14 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 256 (256) - ScannerThreadsTotalWallClockTime: 421.063ms - DelimiterParseTime: 304.580ms - MaterializeTupleTime(*): 38.983ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 348.000ms - ScannerThreadsVoluntaryContextSwitches: 20 (20) - TotalRawHdfsReadTime(*): 28.517ms - TotalReadThroughput: 48.00 MB/sec Averaged Fragment F02:(Total: 669.080ms, non-child: 324.460ms, % non-child: 48.49%) split sizes: min: 112.71 MB, max: 112.71 MB, avg: 112.71 MB, stddev: 0 completion times: min:670.379ms max:670.379ms mean: 670.379ms stddev:0.000ns execution rates: min:168.13 MB/sec max:168.13 MB/sec mean:168.13 MB/sec stddev:0.69 B/sec num instances: 1 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.15 MB (34760260) - PerHostPeakMemUsage: 167.24 MB (175368737) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.696ms - TotalStorageWaitTime: 1.913ms - TotalThreadsInvoluntaryContextSwitches: 705 (705) - TotalThreadsTotalWallClockTime: 1s208ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 964.000ms - TotalThreadsVoluntaryContextSwitches: 265 (265) Fragment Instance Lifecycle Timings: - ExecTime: 591.831ms - ExecTreeExecTime: 10.208ms - OpenTime: 54.083ms - ExecTreeOpenTime: 82.873us - PrepareTime: 23.121ms - ExecTreePrepareTime: 83.921us DataStreamSender (dst_id=8):(Total: 262.502ms, non-child: 262.502ms, % non-child: 100.00%) - BytesSent: 6.12 MB (6416198) - NetworkThroughput(*): 155.47 MB/sec - OverallThroughput: 23.31 MB/sec - PeakMemoryUsage: 148.44 KB (152000) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 223.045ms - TransmitDataRPCTime: 39.356ms - UncompressedRowBatchSize: 9.92 MB (10401764) CodeGen:(Total: 72.515ms, non-child: 72.515ms, % non-child: 100.00%) - CodegenTime: 1.120ms - CompileTime: 4.557ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 48.517ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.909ms HDFS_SCAN_NODE (id=0):(Total: 9.601ms, non-child: 9.601ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 3.81 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.432ms - RowBatchQueuePutWaitTime: 119.547ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 83.32 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 327 (327) - ScannerThreadsTotalWallClockTime: 562.132ms - DelimiterParseTime: 390.735ms - MaterializeTupleTime(*): 42.512ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 380.000ms - ScannerThreadsVoluntaryContextSwitches: 255 (255) - TotalRawHdfsReadTime(*): 28.865ms - TotalReadThroughput: 112.71 MB/sec Fragment F02: Instance 684369c252187019:899cc5d500000004 (host=mikeb-desktop:22000):(Total: 669.080ms, non-child: 324.460ms, % non-child: 48.49%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB MemoryUsage(500.000ms): 24.58 MB, 10.13 MB ThreadUsage(500.000ms): 2, 2 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.15 MB (34760260) - PerHostPeakMemUsage: 167.24 MB (175368737) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.696ms - TotalStorageWaitTime: 1.913ms - TotalThreadsInvoluntaryContextSwitches: 705 (705) - TotalThreadsTotalWallClockTime: 1s208ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 964.000ms - TotalThreadsVoluntaryContextSwitches: 265 (265) Fragment Instance Lifecycle Timings: - ExecTime: 591.831ms - ExecTreeExecTime: 10.208ms - OpenTime: 54.083ms - ExecTreeOpenTime: 82.873us - PrepareTime: 23.121ms - ExecTreePrepareTime: 83.921us DataStreamSender (dst_id=8):(Total: 262.502ms, non-child: 262.502ms, % non-child: 100.00%) - BytesSent: 6.12 MB (6416198) - NetworkThroughput(*): 155.47 MB/sec - OverallThroughput: 23.31 MB/sec - PeakMemoryUsage: 148.44 KB (152000) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 223.045ms - TransmitDataRPCTime: 39.356ms - UncompressedRowBatchSize: 9.92 MB (10401764) CodeGen:(Total: 72.515ms, non-child: 72.515ms, % non-child: 100.00%) - CodegenTime: 1.120ms - CompileTime: 4.557ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 48.517ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.909ms HDFS_SCAN_NODE (id=0):(Total: 9.601ms, non-child: 9.601ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 32.00 MB, 112.71 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 3.81 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 2.432ms - RowBatchQueuePutWaitTime: 119.547ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 83.32 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 327 (327) - ScannerThreadsTotalWallClockTime: 562.132ms - DelimiterParseTime: 390.735ms - MaterializeTupleTime(*): 42.512ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 380.000ms - ScannerThreadsVoluntaryContextSwitches: 255 (255) - TotalRawHdfsReadTime(*): 28.865ms - TotalReadThroughput: 112.71 MB/sec