Query (id=e145c870361a63c6:e4b233c900000000): DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. Summary: Session ID: ca4589f75c58abec:e0b7aecbb7f095b4 Session Type: BEESWAX Start Time: 2017-06-13 14:22:33.842559000 End Time: 2017-06-13 14:22:35.394668000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.9.0-SNAPSHOT DEBUG (build a2281508ff64b9ef3ea730cfd98f6a1c7a497766) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:32802 Default Db: tpch Sql Statement: select count(t1.ps_suppkey) from partsupp t1 LEFT JOIN ( SELECT t2.o_custkey AS coalesce_col FROM orders t2 LEFT JOIN partsupp t3 ON (t2.o_shippriority) = (t3.ps_partkey) ) sq ON ((t1.ps_suppkey) = (sq.coalesce_col)) Coordinator: mikeb-desktop:22000 Query Options (non default): Plan: ---------------- Per-Host Resource Reservation: Memory=272.00MB Per-Host Resource Estimates: Memory=365.43MB F04:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 10:AGGREGATE [FINALIZE] | output: count:merge(t1.ps_suppkey) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | 09:EXCHANGE [UNPARTITIONED] | mem-estimate=0B mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | F03:PLAN FRAGMENT [HASH(t2.o_custkey)] hosts=1 instances=1 05:AGGREGATE | output: count(t1.ps_suppkey) | mem-estimate=10.00MB mem-reservation=0B | tuple-ids=4 row-size=8B cardinality=1 | 04:HASH JOIN [RIGHT OUTER JOIN, PARTITIONED] | hash predicates: t2.o_custkey = (t1.ps_suppkey) | runtime filters: RF000 <- (t1.ps_suppkey) | mem-estimate=6.71MB mem-reservation=136.00MB | tuple-ids=1N,2N,0 row-size=28B cardinality=800000 | |--08:EXCHANGE [HASH((t1.ps_suppkey))] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=0 row-size=8B cardinality=800000 | | | F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | 00:SCAN HDFS [tpch.partsupp t1, RANDOM] | partitions=1/1 files=1 size=112.71MB | stats-rows=800000 extrapolated-rows=disabled | table stats: rows=800000 size=unavailable | column stats: all | mem-estimate=176.00MB mem-reservation=0B | tuple-ids=0 row-size=8B cardinality=800000 | 07:EXCHANGE [HASH(t2.o_custkey)] | mem-estimate=0B mem-reservation=0B | tuple-ids=1,2N row-size=20B cardinality=1500000 | F00:PLAN FRAGMENT [RANDOM] hosts=2 instances=2 03:HASH JOIN [LEFT OUTER JOIN, BROADCAST] | hash predicates: (t2.o_shippriority) = (t3.ps_partkey) | mem-estimate=6.71MB mem-reservation=136.00MB | tuple-ids=1,2N row-size=20B cardinality=1500000 | |--06:EXCHANGE [BROADCAST] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=2 row-size=8B cardinality=800000 | | | F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1 | 02:SCAN HDFS [tpch.partsupp t3, RANDOM] | partitions=1/1 files=1 size=112.71MB | stats-rows=800000 extrapolated-rows=disabled | table stats: rows=800000 size=unavailable | column stats: all | mem-estimate=176.00MB mem-reservation=0B | tuple-ids=2 row-size=8B cardinality=800000 | 01:SCAN HDFS [tpch.orders t2, RANDOM] partitions=1/1 files=1 size=162.56MB runtime filters: RF000 -> t2.o_custkey stats-rows=1500000 extrapolated-rows=disabled table stats: rows=1500000 size=unavailable column stats: all mem-estimate=176.00MB mem-reservation=0B tuple-ids=1 row-size=12B cardinality=1500000 ---------------- Estimated Per-Host Mem: 383178754 Per-Host Memory Reservation: 285212672 Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ----------------------------------------------------------------------------------------------------------------------------- 10:AGGREGATE 1 196.528us 196.528us 1 1 24.00 KB 10.00 MB FINALIZE 09:EXCHANGE 1 38.678us 38.678us 2 1 0 0 UNPARTITIONED 05:AGGREGATE 2 68.671ms 75.556ms 2 1 50.89 KB 10.00 MB 04:HASH JOIN 2 460.980ms 472.610ms 8.23M 800.00K 114.07 MB 6.71 MB RIGHT OUTER JOIN, PARTITIONED |--08:EXCHANGE 2 53.569ms 60.459ms 800.00K 800.00K 0 0 HASH((t1.ps_suppkey)) | 00:SCAN HDFS 1 19.808ms 19.808ms 800.00K 800.00K 33.02 MB 176.00 MB tpch.partsupp t1 07:EXCHANGE 2 91.832ms 101.949ms 1.50M 1.50M 0 0 HASH(t2.o_custkey) 03:HASH JOIN 2 206.506ms 228.767ms 1.50M 1.50M 178.05 MB 6.71 MB LEFT OUTER JOIN, BROADCAST |--06:EXCHANGE 2 95.342ms 97.771ms 800.00K 800.00K 0 0 BROADCAST | 02:SCAN HDFS 1 34.058ms 34.058ms 800.00K 800.00K 33.02 MB 176.00 MB tpch.partsupp t3 01:SCAN HDFS 2 93.905ms 101.139ms 1.50M 1.50M 33.25 MB 176.00 MB tpch.orders t2 Errors: Planner Timeline: 8.843ms - Analysis finished: 3.393ms (3.393ms) - Equivalence classes computed: 3.677ms (283.056us) - Single node plan created: 6.930ms (3.253ms) - Runtime filters computed: 7.174ms (243.771us) - Distributed plan created: 7.486ms (312.038us) - Planning finished: 8.843ms (1.357ms) Query Timeline: 1s555ms - Query submitted: 82.522us (82.522us) - Planning finished: 9.928ms (9.845ms) - Submit for admission: 10.228ms (300.445us) - Completed admission: 10.268ms (39.898us) - Ready to start on 2 backends: 10.613ms (344.824us) - All 2 execution backends (7 fragment instances) started: 11.570ms (956.971us) - First dynamic filter received: 686.032ms (674.462ms) - Rows available: 1s520ms (834.442ms) - First row fetched: 1s548ms (27.802ms) - Unregister query: 1s552ms (3.803ms) - ComputeScanRangeAssignmentTimer: 86.890us ImpalaServer: - ClientFetchWaitTimer: 29.293ms - RowMaterializationTimer: 2.293ms Execution Profile e145c870361a63c6:e4b233c900000000:(Total: 1s512ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 1 REMOTE false 2 (2) N/A N/A true Backend startup latencies: Count: 2, min / max: 0 / 1ms, 25th %-ile: 0, 50th %-ile: 0, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 1 REMOTE false 0 (2) 686.048ms 686.784ms false Per Node Peak Memory Usage: mikeb-desktop:22000(328.56 MB) mikeb-desktop:22001(319.86 MB) - FiltersReceived: 2 (2) - FinalizationTimer: 0.000ns Averaged Fragment F04:(Total: 1s536ms, non-child: 28.274ms, % non-child: 1.84%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1s537ms max:1s537ms mean: 1s537ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.02 KB (32784) - PerHostPeakMemUsage: 328.56 MB (344525277) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s487ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 1s515ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 4 (4) Fragment Instance Lifecycle Timings: - ExecTime: 27.867ms - ExecTreeExecTime: 3.615us - OpenTime: 1s487ms - ExecTreeOpenTime: 1s487ms - PrepareTime: 21.000ms - ExecTreePrepareTime: 135.450us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 154 (154) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 256.00 MB (268435456) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 20.507ms, non-child: 20.507ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 20.010ms AGGREGATION_NODE (id=10):(Total: 1s487ms, non-child: 196.528us, % non-child: 0.01%) - BuildTime: 4.435us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=9):(Total: 1s487ms, non-child: 1s487ms, % non-child: 100.00%) - BytesReceived: 32.00 B (32) - ConvertRowBatchTime: 2.807us - DeserializeRowBatchTimer: 12.770us - FirstBatchArrivalWaitTime: 1s486ms - PeakMemoryUsage: 0 - RowsReturned: 2 (2) - RowsReturnedRate: 1.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Coordinator Fragment F04: Instance e145c870361a63c6:e4b233c900000000 (host=mikeb-desktop:22000):(Total: 1s536ms, non-child: 28.274ms, % non-child: 1.84%) MemoryUsage(500.000ms): 16.00 KB, 16.00 KB, 16.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 32.02 KB (32784) - PerHostPeakMemUsage: 328.56 MB (344525277) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 1s487ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 0 (0) - TotalThreadsTotalWallClockTime: 1s515ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 0.000ns - TotalThreadsVoluntaryContextSwitches: 4 (4) Fragment Instance Lifecycle Timings: - ExecTime: 27.867ms - ExecTreeExecTime: 3.615us - OpenTime: 1s487ms - ExecTreeOpenTime: 1s487ms - PrepareTime: 21.000ms - ExecTreePrepareTime: 135.450us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 154 (154) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 256.00 MB (268435456) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 20.507ms, non-child: 20.507ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 20.010ms AGGREGATION_NODE (id=10):(Total: 1s487ms, non-child: 196.528us, % non-child: 0.01%) - BuildTime: 4.435us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 24.00 KB (24576) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=9):(Total: 1s487ms, non-child: 38.678us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0 - BytesReceived: 32.00 B (32) - ConvertRowBatchTime: 2.807us - DeserializeRowBatchTimer: 12.770us - FirstBatchArrivalWaitTime: 1s486ms - PeakMemoryUsage: 0 - RowsReturned: 2 (2) - RowsReturnedRate: 1.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F03:(Total: 1s508ms, non-child: 9.390ms, % non-child: 0.62%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1s508ms max:1s509ms mean: 1s509ms stddev:499.302us execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 2 - AverageThreadTokens: 1.33 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 112.04 MB (117480205) - PerHostPeakMemUsage: 324.21 MB (339963257) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 937.820ms - TotalNetworkSendTime: 212.383us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 497 (497) - TotalThreadsTotalWallClockTime: 1s978ms - TotalThreadsSysTime: 12.000ms - TotalThreadsUserTime: 954.000ms - TotalThreadsVoluntaryContextSwitches: 738 (738) Fragment Instance Lifecycle Timings: - ExecTime: 324.833us - ExecTreeExecTime: 3.482us - OpenTime: 1s482ms - ExecTreeOpenTime: 1s281ms - PrepareTime: 25.709ms - ExecTreePrepareTime: 292.581us DataStreamSender (dst_id=9):(Total: 44.779us, non-child: 44.779us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 1.02 MB/sec - OverallThroughput: 351.94 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 7.882us - TransmitDataRPCTime: 15.829us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 217.217ms, non-child: 217.217ms, % non-child: 100.00%) - CodegenTime: 417.412us - CompileTime: 18.337ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 60 (60) - NumInstructions: 2.52K (2520) - OptimizationTime: 181.610ms - PeakMemoryUsage: 1.23 MB (1290240) - PrepareTime: 16.730ms AGGREGATION_NODE (id=5):(Total: 1s281ms, non-child: 68.671ms, % non-child: 5.36%) - BuildTime: 63.387ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 50.89 KB (52112) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s212ms, non-child: 460.980ms, % non-child: 38.01%) - BuildRows: 400.00K (400000) - BuildTime: 171.203ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 110.07 MB (115414160) - ProbeRows: 750.00K (750000) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 409.286ms - RowsReturned: 4.11M (4113240) - RowsReturnedRate: 3.39 M/sec Hash Join Builder (join_node_id=4): - BuildRowsPartitionTime: 118.951ms - BuildRowsPartitioned: 400.00K (400000) - GetNewBlockTime: 101.032us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 51.377ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 110.02 MB (115364992) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 323.529ms, non-child: 323.529ms, % non-child: 100.00%) - BytesReceived: 3.06 MB (3208099) - ConvertRowBatchTime: 50.242ms - DeserializeRowBatchTimer: 10.294ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 400.00K (400000) - RowsReturnedRate: 1.24 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 752.237ms, non-child: 752.237ms, % non-child: 100.00%) - BytesReceived: 9.23 MB (9675887) - ConvertRowBatchTime: 81.001ms - DeserializeRowBatchTimer: 33.521ms - FirstBatchArrivalWaitTime: 453.181ms - PeakMemoryUsage: 0 - RowsReturned: 750.00K (750000) - RowsReturnedRate: 999.70 K/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 153 (153) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 248.00 MB (260046848) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns Fragment F03: Instance e145c870361a63c6:e4b233c900000006 (host=mikeb-desktop:22000):(Total: 1s508ms, non-child: 12.097ms, % non-child: 0.80%) MemoryUsage(500.000ms): 1.99 MB, 65.60 MB, 115.16 MB ThreadUsage(500.000ms): 1, 2, 1 - AverageThreadTokens: 1.33 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 116.18 MB (121825541) - PerHostPeakMemUsage: 328.56 MB (344525277) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 887.761ms - TotalNetworkSendTime: 206.309us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 524 (524) - TotalThreadsTotalWallClockTime: 1s958ms - TotalThreadsSysTime: 12.000ms - TotalThreadsUserTime: 984.000ms - TotalThreadsVoluntaryContextSwitches: 713 (713) Fragment Instance Lifecycle Timings: - ExecTime: 350.894us - ExecTreeExecTime: 2.654us - OpenTime: 1s476ms - ExecTreeOpenTime: 1s261ms - PrepareTime: 31.716ms - ExecTreePrepareTime: 306.530us DataStreamSender (dst_id=9):(Total: 40.638us, non-child: 40.638us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 1.26 MB/sec - OverallThroughput: 384.49 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 5.132us - TransmitDataRPCTime: 12.067us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 234.275ms, non-child: 234.275ms, % non-child: 100.00%) - CodegenTime: 509.044us - CompileTime: 19.051ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 60 (60) - NumInstructions: 2.52K (2520) - OptimizationTime: 194.429ms - PeakMemoryUsage: 1.23 MB (1290240) - PrepareTime: 20.299ms AGGREGATION_NODE (id=5):(Total: 1s262ms, non-child: 75.556ms, % non-child: 5.99%) ExecOption: Codegen Enabled - BuildTime: 70.162ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 50.89 KB (52112) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s186ms, non-child: 472.610ms, % non-child: 39.83%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 400.88K (400880) - BuildTime: 166.595ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 114.07 MB (119608464) - ProbeRows: 752.21K (752207) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 418.849ms - RowsReturned: 4.08M (4079520) - RowsReturnedRate: 3.44 M/sec Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 110.214ms - BuildRowsPartitioned: 400.88K (400880) - GetNewBlockTime: 113.158us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 55.101ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 114.02 MB (119559296) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 314.173ms, non-child: 46.679ms, % non-child: 14.86%) BytesReceived(500.000ms): 327.67 KB, 3.07 MB, 3.07 MB - BytesReceived: 3.07 MB (3214558) - ConvertRowBatchTime: 42.981ms - DeserializeRowBatchTimer: 10.539ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 400.88K (400880) - RowsReturnedRate: 1.28 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 714.440ms, non-child: 101.949ms, % non-child: 14.27%) BytesReceived(500.000ms): 0, 194.87 KB, 6.55 MB - BytesReceived: 9.25 MB (9702579) - ConvertRowBatchTime: 91.318ms - DeserializeRowBatchTimer: 33.325ms - FirstBatchArrivalWaitTime: 435.292ms - PeakMemoryUsage: 0 - RowsReturned: 752.21K (752207) - RowsReturnedRate: 1.05 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance e145c870361a63c6:e4b233c900000005 (host=mikeb-desktop:22001):(Total: 1s507ms, non-child: 6.682ms, % non-child: 0.44%) MemoryUsage(500.000ms): 773.19 KB, 67.71 MB, 107.16 MB ThreadUsage(500.000ms): 1, 2, 1 - AverageThreadTokens: 1.33 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 107.89 MB (113134869) - PerHostPeakMemUsage: 319.86 MB (335401237) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 987.880ms - TotalNetworkSendTime: 218.457us - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 470 (470) - TotalThreadsTotalWallClockTime: 1s997ms - TotalThreadsSysTime: 12.000ms - TotalThreadsUserTime: 924.000ms - TotalThreadsVoluntaryContextSwitches: 764 (764) Fragment Instance Lifecycle Timings: - ExecTime: 298.772us - ExecTreeExecTime: 4.310us - OpenTime: 1s487ms - ExecTreeOpenTime: 1s300ms - PrepareTime: 19.702ms - ExecTreePrepareTime: 278.632us BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 153 (153) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 6.68 GB (7171057152) - PeakMemoryUsage: 248.00 MB (260046848) - ScratchBytesRead: 0 - ScratchBytesWritten: 0 - ScratchFileUsedBytes: 0 - ScratchReads: 0 (0) - ScratchWrites: 0 (0) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalReadBlockTime: 0.000ns DataStreamSender (dst_id=9):(Total: 48.921us, non-child: 48.921us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 797.52 KB/sec - OverallThroughput: 319.39 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - RowsReturned: 1 (1) - SerializeBatchTime: 10.633us - TransmitDataRPCTime: 19.592us - UncompressedRowBatchSize: 16.00 B (16) CodeGen:(Total: 200.159ms, non-child: 200.159ms, % non-child: 100.00%) - CodegenTime: 325.780us - CompileTime: 17.622ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 60 (60) - NumInstructions: 2.52K (2520) - OptimizationTime: 168.790ms - PeakMemoryUsage: 1.23 MB (1290240) - PrepareTime: 13.161ms AGGREGATION_NODE (id=5):(Total: 1s300ms, non-child: 61.785ms, % non-child: 4.75%) ExecOption: Codegen Enabled - BuildTime: 56.612ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 50.89 KB (52112) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 1s239ms, non-child: 449.350ms, % non-child: 36.27%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 399.12K (399120) - BuildTime: 175.811ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 106.07 MB (111219856) - ProbeRows: 747.79K (747793) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 399.723ms - RowsReturned: 4.15M (4146960) - RowsReturnedRate: 3.35 M/sec Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 127.688ms - BuildRowsPartitioned: 399.12K (399120) - GetNewBlockTime: 88.906us - HashBuckets: 1.05M (1048576) - HashCollisions: 0 (0) - HashTablesBuildTime: 47.653ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 106.02 MB (111170688) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=8):(Total: 332.886ms, non-child: 60.459ms, % non-child: 18.16%) BytesReceived(500.000ms): 327.76 KB, 3.05 MB, 3.05 MB - BytesReceived: 3.05 MB (3201640) - ConvertRowBatchTime: 57.503ms - DeserializeRowBatchTimer: 10.050ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 399.12K (399120) - RowsReturnedRate: 1.20 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns EXCHANGE_NODE (id=7):(Total: 790.033ms, non-child: 81.715ms, % non-child: 10.34%) BytesReceived(500.000ms): 0, 186.50 KB, 6.52 MB - BytesReceived: 9.20 MB (9649196) - ConvertRowBatchTime: 70.683ms - DeserializeRowBatchTimer: 33.716ms - FirstBatchArrivalWaitTime: 471.070ms - PeakMemoryUsage: 0 - RowsReturned: 747.79K (747793) - RowsReturnedRate: 946.53 K/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1s222ms, non-child: 170.456ms, % non-child: 13.94%) split sizes: min: 34.56 MB, max: 128.00 MB, avg: 81.28 MB, stddev: 46.72 MB completion times: min:978.200ms max:1s469ms mean: 1s223ms stddev:245.577ms execution rates: min:35.33 MB/sec max:87.11 MB/sec mean:61.22 MB/sec stddev:25.89 MB/sec num instances: 2 - AverageThreadTokens: 1.58 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 212.23 MB (222539552) - PerHostPeakMemUsage: 324.21 MB (339963257) - RowsProduced: 750.00K (750000) - TotalNetworkReceiveTime: 3.840ms - TotalNetworkSendTime: 7.408ms - TotalStorageWaitTime: 4.714ms - TotalThreadsInvoluntaryContextSwitches: 482 (482) - TotalThreadsTotalWallClockTime: 1s962ms - TotalThreadsSysTime: 12.000ms - TotalThreadsUserTime: 1s510ms - TotalThreadsVoluntaryContextSwitches: 295 (295) Fragment Instance Lifecycle Timings: - ExecTime: 536.488ms - ExecTreeExecTime: 42.665ms - OpenTime: 657.906ms - ExecTreeOpenTime: 353.021ms - PrepareTime: 28.461ms - ExecTreePrepareTime: 334.330us DataStreamSender (dst_id=7):(Total: 333.224ms, non-child: 333.224ms, % non-child: 100.00%) - BytesSent: 9.23 MB (9675887) - NetworkThroughput(*): 99.45 MB/sec - OverallThroughput: 26.63 MB/sec - PeakMemoryUsage: 47.25 KB (48384) - RowsReturned: 750.00K (750000) - SerializeBatchTime: 236.518ms - TransmitDataRPCTime: 96.533ms - UncompressedRowBatchSize: 15.03 MB (15758072) CodeGen:(Total: 323.491ms, non-child: 323.491ms, % non-child: 100.00%) - CodegenTime: 1.233ms - CompileTime: 23.984ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 280.024ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 19.036ms HASH_JOIN_NODE (id=3):(Total: 395.751ms, non-child: 206.506ms, % non-child: 52.18%) - BuildRows: 800.00K (800000) - BuildTime: 170.658ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 750.00K (750000) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 29.103ms - RowsReturned: 750.00K (750000) - RowsReturnedRate: 1.82 M/sec Hash Join Builder (join_node_id=3): - BuildRowsPartitionTime: 74.492ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 69.826us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 95.880ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 95.342ms, non-child: 95.342ms, % non-child: 100.00%) - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 88.592ms - DeserializeRowBatchTimer: 25.236ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 8.40 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 93.905ms, non-child: 93.905ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 81.31 MB (85258848) - BytesReadDataNodeCache: 0 - BytesReadLocal: 81.31 MB (85258848) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 81.31 MB (85258848) - CachedFileHandlesHitCount: 11 (11) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.08 MB (34687488) - PerReadThreadRawHdfsThroughput: 3.34 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 8.591ms - RowBatchQueuePutWaitTime: 42.552ms - RowsRead: 750.00K (750000) - RowsReturned: 750.00K (750000) - RowsReturnedRate: 7.68 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 161 (161) - ScannerThreadsTotalWallClockTime: 500.565ms - DelimiterParseTime: 391.293ms - MaterializeTupleTime(*): 51.086ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 442.000ms - ScannerThreadsVoluntaryContextSwitches: 226 (226) - TotalRawHdfsReadTime(*): 23.018ms - TotalReadThroughput: 50.67 MB/sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Fragment F00: Instance e145c870361a63c6:e4b233c900000003 (host=mikeb-desktop:22000):(Total: 1s468ms, non-child: 216.314ms, % non-child: 14.73%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB Filter 0 arrival: 676ms MemoryUsage(500.000ms): 5.09 MB, 211.19 MB, 204.41 MB ThreadUsage(500.000ms): 1, 2, 2 - AverageThreadTokens: 1.67 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 212.41 MB (222726432) - PerHostPeakMemUsage: 328.56 MB (344525277) - RowsProduced: 1.18M (1181503) - TotalNetworkReceiveTime: 3.968ms - TotalNetworkSendTime: 4.591ms - TotalStorageWaitTime: 6.366ms - TotalThreadsInvoluntaryContextSwitches: 534 (534) - TotalThreadsTotalWallClockTime: 2s471ms - TotalThreadsSysTime: 20.000ms - TotalThreadsUserTime: 1s992ms - TotalThreadsVoluntaryContextSwitches: 334 (334) Fragment Instance Lifecycle Timings: - ExecTime: 777.259ms - ExecTreeExecTime: 64.340ms - OpenTime: 661.487ms - ExecTreeOpenTime: 363.579ms - PrepareTime: 29.712ms - ExecTreePrepareTime: 302.177us DataStreamSender (dst_id=7):(Total: 507.382ms, non-child: 507.382ms, % non-child: 100.00%) - BytesSent: 14.54 MB (15249826) - NetworkThroughput(*): 93.19 MB/sec - OverallThroughput: 28.66 MB/sec - PeakMemoryUsage: 47.25 KB (48384) - RowsReturned: 1.18M (1181503) - SerializeBatchTime: 351.079ms - TransmitDataRPCTime: 156.058ms - UncompressedRowBatchSize: 23.67 MB (24824275) CodeGen:(Total: 317.166ms, non-child: 317.166ms, % non-child: 100.00%) - CodegenTime: 1.057ms - CompileTime: 26.895ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 270.231ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 19.682ms HASH_JOIN_NODE (id=3):(Total: 427.674ms, non-child: 228.767ms, % non-child: 53.49%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 800.00K (800000) - BuildTime: 177.868ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 1.18M (1181503) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 42.599ms - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 2.76 M/sec Hash Join Builder (join_node_id=3): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 77.463ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 71.810us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 100.109ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 97.771ms, non-child: 97.771ms, % non-child: 100.00%) BytesReceived(500.000ms): 1.26 MB, 4.59 MB, 4.59 MB - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 90.713ms - DeserializeRowBatchTimer: 25.369ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 8.18 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 101.139ms, non-child: 101.139ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/128.00 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 81ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0, 32.00 MB, 104.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 128.06 MB (134283264) - BytesReadDataNodeCache: 0 - BytesReadLocal: 128.06 MB (134283264) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 128.06 MB (134283264) - CachedFileHandlesHitCount: 17 (17) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 2 (2) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.25 MB (34870272) - PerReadThreadRawHdfsThroughput: 3.55 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 13.587ms - RowBatchQueuePutWaitTime: 46.135ms - RowsRead: 1.18M (1181503) - RowsReturned: 1.18M (1181503) - RowsReturnedRate: 11.68 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 221 (221) - ScannerThreadsTotalWallClockTime: 754.713ms - DelimiterParseTime: 603.720ms - MaterializeTupleTime(*): 78.960ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 680.000ms - ScannerThreadsVoluntaryContextSwitches: 254 (254) - TotalRawHdfsReadTime(*): 35.247ms - TotalReadThroughput: 69.33 MB/sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Instance e145c870361a63c6:e4b233c900000002 (host=mikeb-desktop:22001):(Total: 977.311ms, non-child: 124.599ms, % non-child: 12.75%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB Filter 0 arrival: 676ms MemoryUsage(500.000ms): 5.09 MB, 211.50 MB ThreadUsage(500.000ms): 1, 2 - AverageThreadTokens: 1.50 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 212.05 MB (222352672) - PerHostPeakMemUsage: 319.86 MB (335401237) - RowsProduced: 318.50K (318497) - TotalNetworkReceiveTime: 3.712ms - TotalNetworkSendTime: 10.226ms - TotalStorageWaitTime: 3.062ms - TotalThreadsInvoluntaryContextSwitches: 430 (430) - TotalThreadsTotalWallClockTime: 1s454ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 1s028ms - TotalThreadsVoluntaryContextSwitches: 256 (256) Fragment Instance Lifecycle Timings: - ExecTime: 295.717ms - ExecTreeExecTime: 20.991ms - OpenTime: 654.324ms - ExecTreeOpenTime: 342.462ms - PrepareTime: 27.209ms - ExecTreePrepareTime: 366.484us DataStreamSender (dst_id=7):(Total: 159.067ms, non-child: 159.067ms, % non-child: 100.00%) - BytesSent: 3.91 MB (4101949) - NetworkThroughput(*): 105.70 MB/sec - OverallThroughput: 24.59 MB/sec - PeakMemoryUsage: 47.25 KB (48384) - RowsReturned: 318.50K (318497) - SerializeBatchTime: 121.958ms - TransmitDataRPCTime: 37.008ms - UncompressedRowBatchSize: 6.38 MB (6691869) CodeGen:(Total: 329.816ms, non-child: 329.816ms, % non-child: 100.00%) - CodegenTime: 1.409ms - CompileTime: 21.073ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 74 (74) - NumInstructions: 2.98K (2981) - OptimizationTime: 289.818ms - PeakMemoryUsage: 1.46 MB (1526272) - PrepareTime: 18.389ms HASH_JOIN_NODE (id=3):(Total: 363.828ms, non-child: 184.245ms, % non-child: 50.64%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 800.00K (800000) - BuildTime: 163.448ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 178.05 MB (186703104) - ProbeRows: 318.50K (318497) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 15.607ms - RowsReturned: 318.50K (318497) - RowsReturnedRate: 875.40 K/sec Hash Join Builder (join_node_id=3): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled - BuildRowsPartitionTime: 71.520ms - BuildRowsPartitioned: 800.00K (800000) - GetNewBlockTime: 67.843us - HashBuckets: 2.10M (2097152) - HashCollisions: 0 (0) - HashTablesBuildTime: 91.651ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 178.02 MB (186668160) - PinTime: 0.000ns - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) - UnpinTime: 0.000ns EXCHANGE_NODE (id=6):(Total: 92.913ms, non-child: 92.913ms, % non-child: 100.00%) BytesReceived(500.000ms): 1.26 MB, 4.59 MB - BytesReceived: 4.59 MB (4810770) - ConvertRowBatchTime: 86.471ms - DeserializeRowBatchTimer: 25.104ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 0 - RowsReturned: 800.00K (800000) - RowsReturnedRate: 8.61 M/sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns HDFS_SCAN_NODE (id=1):(Total: 86.672ms, non-child: 86.672ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/34.56 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Runtime filters: All filters arrived. Waited 80ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0, 32.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 34.56 MB (36234433) - BytesReadDataNodeCache: 0 - BytesReadLocal: 34.56 MB (36234433) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 34.56 MB (36234433) - CachedFileHandlesHitCount: 5 (5) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 32.91 MB (34504704) - PerReadThreadRawHdfsThroughput: 3.13 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 3.596ms - RowBatchQueuePutWaitTime: 38.970ms - RowsRead: 318.50K (318497) - RowsReturned: 318.50K (318497) - RowsReturnedRate: 3.67 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 101 (101) - ScannerThreadsTotalWallClockTime: 246.417ms - DelimiterParseTime: 178.865ms - MaterializeTupleTime(*): 23.212ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 204.000ms - ScannerThreadsVoluntaryContextSwitches: 199 (199) - TotalRawHdfsReadTime(*): 10.789ms - TotalReadThroughput: 32.00 MB/sec Filter 0 (1.00 MB): - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) Averaged Fragment F01:(Total: 474.258ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 112.71 MB, max: 112.71 MB, avg: 112.71 MB, stddev: 0 completion times: min:475.387ms max:475.387ms mean: 475.387ms stddev:0.000ns execution rates: min:237.09 MB/sec max:237.09 MB/sec mean:237.09 MB/sec stddev:0.09 B/sec num instances: 1 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.06 MB (34670148) - PerHostPeakMemUsage: 83.88 MB (87955305) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 102.930ms - TotalStorageWaitTime: 9.270ms - TotalThreadsInvoluntaryContextSwitches: 569 (569) - TotalThreadsTotalWallClockTime: 843.454ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 632.000ms - TotalThreadsVoluntaryContextSwitches: 432 (432) Fragment Instance Lifecycle Timings: - ExecTime: 401.523ms - ExecTreeExecTime: 28.542ms - OpenTime: 50.269ms - ExecTreeOpenTime: 65.275us - PrepareTime: 22.423ms - ExecTreePrepareTime: 3.169ms DataStreamSender (dst_id=6):(Total: 382.003ms, non-child: 382.003ms, % non-child: 100.00%) - BytesSent: 9.18 MB (9621540) - NetworkThroughput(*): 62.80 MB/sec - OverallThroughput: 24.02 MB/sec - PeakMemoryUsage: 28.44 KB (29120) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 235.742ms - TransmitDataRPCTime: 146.107ms - UncompressedRowBatchSize: 19.84 MB (20806256) CodeGen:(Total: 68.726ms, non-child: 68.726ms, % non-child: 100.00%) - CodegenTime: 1.013ms - CompileTime: 3.543ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 45.722ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.992ms HDFS_SCAN_NODE (id=2):(Total: 34.058ms, non-child: 34.058ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 3.87 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 20.315ms - RowBatchQueuePutWaitTime: 8.254ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 23.49 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 163 (163) - ScannerThreadsTotalWallClockTime: 391.671ms - DelimiterParseTime: 326.746ms - MaterializeTupleTime(*): 41.601ms - ScannerThreadsSysTime: 4.000ms - ScannerThreadsUserTime: 352.000ms - ScannerThreadsVoluntaryContextSwitches: 13 (13) - TotalRawHdfsReadTime(*): 28.448ms - TotalReadThroughput: 112.00 MB/sec Fragment F01: Instance e145c870361a63c6:e4b233c900000001 (host=mikeb-desktop:22000):(Total: 474.258ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB MemoryUsage(500.000ms): 25.05 MB ThreadUsage(500.000ms): 2 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.06 MB (34670148) - PerHostPeakMemUsage: 83.88 MB (87955305) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 102.930ms - TotalStorageWaitTime: 9.270ms - TotalThreadsInvoluntaryContextSwitches: 569 (569) - TotalThreadsTotalWallClockTime: 843.454ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 632.000ms - TotalThreadsVoluntaryContextSwitches: 432 (432) Fragment Instance Lifecycle Timings: - ExecTime: 401.523ms - ExecTreeExecTime: 28.542ms - OpenTime: 50.269ms - ExecTreeOpenTime: 65.275us - PrepareTime: 22.423ms - ExecTreePrepareTime: 3.169ms DataStreamSender (dst_id=6):(Total: 382.003ms, non-child: 382.003ms, % non-child: 100.00%) - BytesSent: 9.18 MB (9621540) - NetworkThroughput(*): 62.80 MB/sec - OverallThroughput: 24.02 MB/sec - PeakMemoryUsage: 28.44 KB (29120) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 235.742ms - TransmitDataRPCTime: 146.107ms - UncompressedRowBatchSize: 19.84 MB (20806256) CodeGen:(Total: 68.726ms, non-child: 68.726ms, % non-child: 100.00%) - CodegenTime: 1.013ms - CompileTime: 3.543ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 45.722ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.992ms HDFS_SCAN_NODE (id=2):(Total: 34.058ms, non-child: 34.058ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 56.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 3.87 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 20.315ms - RowBatchQueuePutWaitTime: 8.254ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 23.49 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 163 (163) - ScannerThreadsTotalWallClockTime: 391.671ms - DelimiterParseTime: 326.746ms - MaterializeTupleTime(*): 41.601ms - ScannerThreadsSysTime: 4.000ms - ScannerThreadsUserTime: 352.000ms - ScannerThreadsVoluntaryContextSwitches: 13 (13) - TotalRawHdfsReadTime(*): 28.448ms - TotalReadThroughput: 112.00 MB/sec Averaged Fragment F02:(Total: 670.037ms, non-child: 328.550ms, % non-child: 49.03%) split sizes: min: 112.71 MB, max: 112.71 MB, avg: 112.71 MB, stddev: 0 completion times: min:674.477ms max:674.477ms mean: 674.477ms stddev:0.000ns execution rates: min:167.11 MB/sec max:167.11 MB/sec mean:167.11 MB/sec stddev:0.68 B/sec num instances: 1 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.15 MB (34760260) - PerHostPeakMemUsage: 206.41 MB (216433957) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 469.921us - TotalStorageWaitTime: 13.090ms - TotalThreadsInvoluntaryContextSwitches: 559 (559) - TotalThreadsTotalWallClockTime: 1s219ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 924.000ms - TotalThreadsVoluntaryContextSwitches: 296 (296) Fragment Instance Lifecycle Timings: - ExecTime: 597.804ms - ExecTreeExecTime: 20.202ms - OpenTime: 53.195ms - ExecTreeOpenTime: 87.786us - PrepareTime: 18.994ms - ExecTreePrepareTime: 85.736us DataStreamSender (dst_id=8):(Total: 250.374ms, non-child: 250.374ms, % non-child: 100.00%) - BytesSent: 6.12 MB (6416198) - NetworkThroughput(*): 165.58 MB/sec - OverallThroughput: 24.44 MB/sec - PeakMemoryUsage: 148.44 KB (152000) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 213.319ms - TransmitDataRPCTime: 36.954ms - UncompressedRowBatchSize: 9.92 MB (10401764) CodeGen:(Total: 71.303ms, non-child: 71.303ms, % non-child: 100.00%) - CodegenTime: 956.715us - CompileTime: 5.452ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 46.701ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.689ms HDFS_SCAN_NODE (id=0):(Total: 19.808ms, non-child: 19.808ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 4.10 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 12.029ms - RowBatchQueuePutWaitTime: 162.923ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 40.39 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 187 (187) - ScannerThreadsTotalWallClockTime: 568.733ms - DelimiterParseTime: 343.225ms - MaterializeTupleTime(*): 36.931ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 364.000ms - ScannerThreadsVoluntaryContextSwitches: 289 (289) - TotalRawHdfsReadTime(*): 26.829ms - TotalReadThroughput: 80.00 MB/sec Fragment F02: Instance e145c870361a63c6:e4b233c900000004 (host=mikeb-desktop:22000):(Total: 670.037ms, non-child: 328.550ms, % non-child: 49.03%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB MemoryUsage(500.000ms): 25.12 MB ThreadUsage(500.000ms): 2 - AverageThreadTokens: 2.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 33.15 MB (34760260) - PerHostPeakMemUsage: 206.41 MB (216433957) - RowsProduced: 800.00K (800000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 469.921us - TotalStorageWaitTime: 13.090ms - TotalThreadsInvoluntaryContextSwitches: 559 (559) - TotalThreadsTotalWallClockTime: 1s219ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 924.000ms - TotalThreadsVoluntaryContextSwitches: 296 (296) Fragment Instance Lifecycle Timings: - ExecTime: 597.804ms - ExecTreeExecTime: 20.202ms - OpenTime: 53.195ms - ExecTreeOpenTime: 87.786us - PrepareTime: 18.994ms - ExecTreePrepareTime: 85.736us DataStreamSender (dst_id=8):(Total: 250.374ms, non-child: 250.374ms, % non-child: 100.00%) - BytesSent: 6.12 MB (6416198) - NetworkThroughput(*): 165.58 MB/sec - OverallThroughput: 24.44 MB/sec - PeakMemoryUsage: 148.44 KB (152000) - RowsReturned: 800.00K (800000) - SerializeBatchTime: 213.319ms - TransmitDataRPCTime: 36.954ms - UncompressedRowBatchSize: 9.92 MB (10401764) CodeGen:(Total: 71.303ms, non-child: 71.303ms, % non-child: 100.00%) - CodegenTime: 956.715us - CompileTime: 5.452ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.98 MB (2077340) - NumFunctions: 15 (15) - NumInstructions: 300 (300) - OptimizationTime: 46.701ms - PeakMemoryUsage: 150.00 KB (153600) - PrepareTime: 18.689ms HDFS_SCAN_NODE (id=0):(Total: 19.808ms, non-child: 19.808ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:1/112.71 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 40.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 1.00 - BytesRead: 112.71 MB (118184616) - BytesReadDataNodeCache: 0 - BytesReadLocal: 112.71 MB (118184616) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 112.71 MB (118184616) - CachedFileHandlesHitCount: 15 (15) - CachedFileHandlesMissCount: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 1 (1) - PeakMemoryUsage: 33.02 MB (34620416) - PerReadThreadRawHdfsThroughput: 4.10 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 12.029ms - RowBatchQueuePutWaitTime: 162.923ms - RowsRead: 800.00K (800000) - RowsReturned: 800.00K (800000) - RowsReturnedRate: 40.39 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 187 (187) - ScannerThreadsTotalWallClockTime: 568.733ms - DelimiterParseTime: 343.225ms - MaterializeTupleTime(*): 36.931ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 364.000ms - ScannerThreadsVoluntaryContextSwitches: 289 (289) - TotalRawHdfsReadTime(*): 26.829ms - TotalReadThroughput: 80.00 MB/sec