Query (id=98428cead46af6f9:f8d48d1e00000000): Summary: Session ID: 5b4aaca21a650825:b6cbe1293df6fcba Session Type: BEESWAX Start Time: 2017-12-21 09:00:06.155446000 End Time: 2017-12-21 09:00:06.770274000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.11.0-RELEASE RELEASE (build Could not obtain git hash) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:45245 Default Db: default Sql Statement: with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem) select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a join (select * from l LIMIT 2000000) b on a.l_orderkey = -b.l_orderkey; Coordinator: mikeb-desktop:22000 Query Options (set by configuration): Query Options (set by configuration and planner): MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=17.00MB Per-Host Resource Estimates: Memory=555.00MB F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=27.00MB mem-reservation=17.00MB PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 05:AGGREGATE [FINALIZE] | output: count(*) | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | tuple-ids=7 row-size=8B cardinality=1 | 04:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: a.l_orderkey = -1 * l_orderkey | fk/pk conjuncts: assumed fk/pk | runtime filters: RF000[bloom] <- -1 * l_orderkey | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB | tuple-ids=0,4 row-size=16B cardinality=1 | |--08:EXCHANGE [UNPARTITIONED] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B | 07:EXCHANGE [UNPARTITIONED] | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F04:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | Per-Host Resources: mem-estimate=264.00MB mem-reservation=0B | 01:UNION | | pass-through-operands: all | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | |--03:SCAN HDFS [tpch.lineitem, RANDOM] | | partitions=1/1 files=1 size=718.94MB | | stored statistics: | | table: rows=6001215 size=718.94MB | | columns: all | | extrapolated-rows=disabled | | mem-estimate=264.00MB mem-reservation=0B | | tuple-ids=3 row-size=8B cardinality=6001215 | | | 02:SCAN HDFS [tpch.lineitem, RANDOM] | partitions=1/1 files=1 size=718.94MB | stored statistics: | table: rows=6001215 size=718.94MB | columns: all | extrapolated-rows=disabled | mem-estimate=264.00MB mem-reservation=0B | tuple-ids=2 row-size=8B cardinality=6001215 | 06:EXCHANGE [UNPARTITIONED] | limit: 1 | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=8B cardinality=1 | F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 Per-Host Resources: mem-estimate=264.00MB mem-reservation=0B 00:SCAN HDFS [tpch.lineitem a, RANDOM] partitions=1/1 files=1 size=718.94MB runtime filters: RF000[bloom] -> a.l_orderkey stored statistics: table: rows=6001215 size=718.94MB columns: all extrapolated-rows=disabled limit: 1 mem-estimate=264.00MB mem-reservation=0B tuple-ids=0 row-size=8B cardinality=1 ---------------- Estimated Per-Host Mem: 581959680 Per Host Min Reservation: mikeb-desktop:22000(17.00 MB) mikeb-desktop:22001(0) mikeb-desktop:22002(0) Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------ 05:AGGREGATE 1 32.485us 32.485us 1 1 28.00 KB 10.00 MB FINALIZE 04:HASH JOIN 1 786.367us 786.367us 0 1 141.06 MB 17.00 MB INNER JOIN, BROADCAST |--08:EXCHANGE 1 35.669ms 35.669ms 2.00M 2.00M 0 0 UNPARTITIONED | 07:EXCHANGE 1 120.253ms 120.253ms 2.00M 2.00M 0 0 UNPARTITIONED | 01:UNION 3 778.946us 936.628us 5.92M 2.00M 0 0 | |--03:SCAN HDFS 3 7.984ms 23.845ms 275.46K 6.00M 56.05 MB 264.00 MB tpch.lineitem | 02:SCAN HDFS 3 204.969ms 256.911ms 5.65M 6.00M 57.05 MB 264.00 MB tpch.lineitem 06:EXCHANGE 1 316.260ms 316.260ms 1 1 0 0 UNPARTITIONED 00:SCAN HDFS 3 324.398ms 529.136ms 2 1 56.08 MB 264.00 MB tpch.lineitem a Errors: Planner Timeline: 4.397ms - Analysis finished: 1.982ms (1.982ms) - Value transfer graph computed: 2.067ms (85.230us) - Single node plan created: 3.106ms (1.038ms) - Runtime filters computed: 3.312ms (206.270us) - Distributed plan created: 3.352ms (40.355us) - Planning finished: 4.397ms (1.045ms) Query Timeline: 616.157ms - Query submitted: 17.063us (17.063us) - Planning finished: 5.275ms (5.258ms) - Submit for admission: 5.373ms (97.513us) - Completed admission: 5.395ms (22.366us) - Ready to start on 3 backends: 5.467ms (72.011us) - All 3 execution backends (8 fragment instances) started: 5.841ms (373.774us) - First dynamic filter received: 325.154ms (319.313ms) - Rows available: 511.831ms (186.676ms) - First row fetched: 542.494ms (30.663ms) - Unregister query: 614.829ms (72.335ms) - ComputeScanRangeAssignmentTimer: 24.440us ImpalaServer: - ClientFetchWaitTimer: 33.506ms - RowMaterializationTimer: 69.481ms Execution Profile 98428cead46af6f9:f8d48d1e00000000:(Total: 575.809ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 0 REMOTE false 1 (1) N/A N/A true Backend startup latencies: Count: 3, min / max: 0 / 0, 25th %-ile: 0, 50th %-ile: 0, 75th %-ile: 0, 90th %-ile: 0, 95th %-ile: 0, 99.9th %-ile: 0 Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 0 REMOTE false 0 (1) 325.155ms 325.156ms false Per Node Peak Memory Usage: mikeb-desktop:22000(247.23 MB) mikeb-desktop:22001(105.24 MB) mikeb-desktop:22002(81.15 MB) - FiltersReceived: 1 (1) - FinalizationTimer: 0.000ns - NumBackends: 3 (3) - NumFragmentInstances: 8 (8) - NumFragments: 4 (4) Averaged Fragment F01:(Total: 536.666ms, non-child: 42.562ms, % non-child: 7.93%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:536.992ms max:536.992ms mean: 536.992ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 142.09 MB (148993814) - PeakReservation: 141.00 MB (147849216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.23 MB (259241492) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 329.310ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 165 (165) - TotalThreadsTotalWallClockTime: 763.352ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 320.000ms - TotalThreadsVoluntaryContextSwitches: 238 (238) Fragment Instance Lifecycle Timings: - ExecTime: 30.715ms - ExecTreeExecTime: 1.659us - OpenTime: 479.354ms - ExecTreeOpenTime: 316.627ms - PrepareTime: 26.548ms - ExecTreePrepareTime: 80.007us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 177.398ms, non-child: 177.398ms, % non-child: 100.00%) - CodegenTime: 220.280us - CompileTime: 49.384ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 87 (87) - NumInstructions: 2.81K (2815) - OptimizationTime: 112.048ms - PeakMemoryUsage: 1.37 MB (1441280) - PrepareTime: 15.560ms AGGREGATION_NODE (id=5):(Total: 316.706ms, non-child: 32.485us, % non-child: 0.01%) - BuildTime: 4.391us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 28.00 KB (28672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 316.674ms, non-child: 786.367us, % non-child: 0.25%) - BuildRows: 2.00M (2000000) - BuildTime: 216.349ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 141.06 MB (147908864) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 966.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 67.311us - CumulativeAllocationBytes: 141.00 MB (147849216) - CumulativeAllocations: 93 (93) - PeakReservation: 141.00 MB (147849216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 141.00 MB (147849216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=4): - BuildRowsPartitionTime: 91.569ms - BuildRowsPartitioned: 2.00M (2000000) - HashBuckets: 4.19M (4194304) - HashCollisions: 0 (0) - HashTablesBuildTime: 123.966ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=8):(Total: 35.669ms, non-child: 35.669ms, % non-child: 100.00%) - ConvertRowBatchTime: 21.864ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 56.07 M/sec DataStreamReceiver: - BytesReceived: 10.91 MB (11440692) - DeserializeRowBatchTimer: 16.717ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 6.401ms - SendersBlockedTotalTimer(*): 6.402ms EXCHANGE_NODE (id=6):(Total: 316.260ms, non-child: 316.260ms, % non-child: 100.00%) - ConvertRowBatchTime: 1.627us - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec DataStreamReceiver: - BytesReceived: 17.00 B (17) - DeserializeRowBatchTimer: 1.539us - FirstBatchArrivalWaitTime: 316.256ms - PeakMemoryUsage: 4.01 KB (4104) - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Coordinator Fragment F01: Instance 98428cead46af6f9:f8d48d1e00000000 (host=mikeb-desktop:22000):(Total: 536.666ms, non-child: 42.562ms, % non-child: 7.93%) MemoryUsage(500.000ms): 113.08 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 142.09 MB (148993814) - PeakReservation: 141.00 MB (147849216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.23 MB (259241492) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 329.310ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 165 (165) - TotalThreadsTotalWallClockTime: 763.352ms - TotalThreadsSysTime: 8.000ms - TotalThreadsUserTime: 320.000ms - TotalThreadsVoluntaryContextSwitches: 238 (238) Fragment Instance Lifecycle Timings: - ExecTime: 30.715ms - ExecTreeExecTime: 1.659us - OpenTime: 479.354ms - ExecTreeOpenTime: 316.627ms - PrepareTime: 26.548ms - ExecTreePrepareTime: 80.007us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 177.398ms, non-child: 177.398ms, % non-child: 100.00%) - CodegenTime: 220.280us - CompileTime: 49.384ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 87 (87) - NumInstructions: 2.81K (2815) - OptimizationTime: 112.048ms - PeakMemoryUsage: 1.37 MB (1441280) - PrepareTime: 15.560ms AGGREGATION_NODE (id=5):(Total: 316.706ms, non-child: 32.485us, % non-child: 0.01%) ExecOption: Codegen Enabled - BuildTime: 4.391us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 28.00 KB (28672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 316.674ms, non-child: 786.367us, % non-child: 0.25%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2.00M (2000000) - BuildTime: 216.349ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 141.06 MB (147908864) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 966.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 67.311us - CumulativeAllocationBytes: 141.00 MB (147849216) - CumulativeAllocations: 93 (93) - PeakReservation: 141.00 MB (147849216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 141.00 MB (147849216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 91.569ms - BuildRowsPartitioned: 2.00M (2000000) - HashBuckets: 4.19M (4194304) - HashCollisions: 0 (0) - HashTablesBuildTime: 123.966ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=8):(Total: 35.669ms, non-child: 35.669ms, % non-child: 100.00%) - ConvertRowBatchTime: 21.864ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 56.07 M/sec DataStreamReceiver: BytesReceived(500.000ms): 10.91 MB - BytesReceived: 10.91 MB (11440692) - DeserializeRowBatchTimer: 16.717ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 6.401ms - SendersBlockedTotalTimer(*): 6.402ms EXCHANGE_NODE (id=6):(Total: 316.260ms, non-child: 316.260ms, % non-child: 100.00%) - ConvertRowBatchTime: 1.627us - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec DataStreamReceiver: BytesReceived(500.000ms): 0 - BytesReceived: 17.00 B (17) - DeserializeRowBatchTimer: 1.539us - FirstBatchArrivalWaitTime: 316.256ms - PeakMemoryUsage: 4.01 KB (4104) - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 370.255ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 206.94 MB, max: 256.00 MB, avg: 239.65 MB, stddev: 23.13 MB completion times: min:506.172ms max:608.815ms mean: 572.933ms stddev:47.222ms execution rates: min:408.83 MB/sec max:422.89 MB/sec mean:417.76 MB/sec stddev:6.37 MB/sec num instances: 3 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.09 MB (59863595) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 93.20 MB (97722118) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 83.916us - TotalStorageWaitTime: 29.965ms - TotalThreadsInvoluntaryContextSwitches: 588 (588) - TotalThreadsTotalWallClockTime: 848.308ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 210.000ms - TotalThreadsVoluntaryContextSwitches: 26 (26) Fragment Instance Lifecycle Timings: - ExecTime: 486.616ms - ExecTreeExecTime: 486.484ms - OpenTime: 50.780ms - ExecTreeOpenTime: 25.694us - PrepareTime: 17.951ms - ExecTreePrepareTime: 37.775us DataStreamSender (dst_id=6):(Total: 1.518ms, non-child: 1.518ms, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 1.50 MB/sec - OverallThroughput: 202.77 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 3.550us - TransmitDataRPCTime: 12.843us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 63.769ms, non-child: 63.769ms, % non-child: 100.00%) - CodegenTime: 1.239ms - CompileTime: 15.067ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 35.002ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 13.269ms HDFS_SCAN_NODE (id=0):(Total: 486.598ms, non-child: 486.598ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 148.00 MB (155189248) - BytesReadDataNodeCache: 0 - BytesReadLocal: 148.00 MB (155189248) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 148.00 MB (155189248) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.07 MB (58790912) - PerReadThreadRawHdfsThroughput: 2.14 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 221.387ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 930.67K (930672) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 576 (576) - ScannerThreadsTotalWallClockTime: 310.918ms - DelimiterParseTime: 395.023ms - MaterializeTupleTime(*): 364.244us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 166.000ms - ScannerThreadsVoluntaryContextSwitches: 7 (7) - TotalRawHdfsReadTime(*): 68.004ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Fragment F00: Instance 98428cead46af6f9:f8d48d1e00000003 (host=mikeb-desktop:22001):(Total: 605.096ms, non-child: 6.469ms, % non-child: 1.07%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Filter 0 arrival: 321ms MemoryUsage(500.000ms): 23.54 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.08 MB (59854379) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 105.24 MB (110357254) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 74.678us - TotalStorageWaitTime: 40.129ms - TotalThreadsInvoluntaryContextSwitches: 341 (341) - TotalThreadsTotalWallClockTime: 851.236ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 180.000ms - TotalThreadsVoluntaryContextSwitches: 22 (22) Fragment Instance Lifecycle Timings: - ExecTime: 529.179ms - ExecTreeExecTime: 529.035ms - OpenTime: 53.868ms - ExecTreeOpenTime: 23.137us - PrepareTime: 22.001ms - ExecTreePrepareTime: 35.220us DataStreamSender (dst_id=6):(Total: 2.995ms, non-child: 2.995ms, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 922.26 KB/sec - OverallThroughput: 5.54 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 4.972us - TransmitDataRPCTime: 18.001us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 66.494ms, non-child: 66.494ms, % non-child: 100.00%) - CodegenTime: 1.259ms - CompileTime: 12.417ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 40.734ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 12.908ms HDFS_SCAN_NODE (id=0):(Total: 529.136ms, non-child: 529.136ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: All filters arrived. Waited 261ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 152.00 MB (159383552) - BytesReadDataNodeCache: 0 - BytesReadLocal: 152.00 MB (159383552) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 152.00 MB (159383552) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.06 MB (58781696) - PerReadThreadRawHdfsThroughput: 2.33 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 267.922ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 988.34K (988342) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 334 (334) - ScannerThreadsTotalWallClockTime: 268.196ms - DelimiterParseTime: 484.262ms - MaterializeTupleTime(*): 340.081us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 140.000ms - ScannerThreadsVoluntaryContextSwitches: 4 (4) - TotalRawHdfsReadTime(*): 63.638ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Instance 98428cead46af6f9:f8d48d1e00000001 (host=mikeb-desktop:22002):(Total: 505.670ms, non-child: 525.259us, % non-child: 0.10%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB Filter 0 arrival: 322ms MemoryUsage(500.000ms): 23.54 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.10 MB (59872811) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 81.15 MB (85086982) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 93.155us - TotalStorageWaitTime: 19.801ms - TotalThreadsInvoluntaryContextSwitches: 836 (836) - TotalThreadsTotalWallClockTime: 845.379ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 240.000ms - TotalThreadsVoluntaryContextSwitches: 30 (30) Fragment Instance Lifecycle Timings: - ExecTime: 444.053ms - ExecTreeExecTime: 443.933ms - OpenTime: 47.693ms - ExecTreeOpenTime: 28.251us - PrepareTime: 13.901ms - ExecTreePrepareTime: 40.331us DataStreamSender (dst_id=6):(Total: 41.503us, non-child: 41.503us, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 2.11 MB/sec - OverallThroughput: 400.01 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 2.128us - TransmitDataRPCTime: 7.686us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 61.043ms, non-child: 61.043ms, % non-child: 100.00%) - CodegenTime: 1.219ms - CompileTime: 17.717ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 29.270ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 13.631ms HDFS_SCAN_NODE (id=0):(Total: 444.060ms, non-child: 444.060ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: All filters arrived. Waited 265ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 144.00 MB (150994944) - BytesReadDataNodeCache: 0 - BytesReadLocal: 144.00 MB (150994944) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 144.00 MB (150994944) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.08 MB (58800128) - PerReadThreadRawHdfsThroughput: 1.94 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 174.851ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 873.00K (873002) - RowsReturned: 1 (1) - RowsReturnedRate: 2.00 /sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 819 (819) - ScannerThreadsTotalWallClockTime: 353.641ms - DelimiterParseTime: 305.783ms - MaterializeTupleTime(*): 388.408us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 192.000ms - ScannerThreadsVoluntaryContextSwitches: 10 (10) - TotalRawHdfsReadTime(*): 72.371ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Instance 98428cead46af6f9:f8d48d1e00000002 (host=mikeb-desktop:22000): Averaged Fragment F05:(Total: 317.894ms, non-child: 18.871ms, % non-child: 5.94%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:318.265ms max:318.265ms mean: 318.265ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 10.04 MB (10528992) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 100.21 MB (105078516) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 88.508ms - TotalNetworkSendTime: 77.565ms - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1.19K (1190) - TotalThreadsTotalWallClockTime: 296.133ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 92.000ms - TotalThreadsVoluntaryContextSwitches: 1.30K (1299) Fragment Instance Lifecycle Timings: - ExecTime: 237.536ms - ExecTreeExecTime: 62.175ms - OpenTime: 58.605ms - ExecTreeOpenTime: 58.596ms - PrepareTime: 21.605ms - ExecTreePrepareTime: 13.056us DataStreamSender (dst_id=8):(Total: 159.526ms, non-child: 159.526ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11440692) - NetworkThroughput(*): 145.73 MB/sec - OverallThroughput: 68.39 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 82.260ms - TransmitDataRPCTime: 74.868ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 19.242ms, non-child: 19.242ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 18.943ms EXCHANGE_NODE (id=7):(Total: 120.253ms, non-child: 120.253ms, % non-child: 100.00%) - ConvertRowBatchTime: 30.454ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 16.63 M/sec DataStreamReceiver: - BytesReceived: 14.29 MB (14987650) - DeserializeRowBatchTimer: 23.775ms - FirstBatchArrivalWaitTime: 58.592ms - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 151.835ms - SendersBlockedTotalTimer(*): 367.944ms Fragment F05: Instance 98428cead46af6f9:f8d48d1e00000007 (host=mikeb-desktop:22000):(Total: 317.894ms, non-child: 18.871ms, % non-child: 5.94%) - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 10.04 MB (10528992) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 100.21 MB (105078516) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 88.508ms - TotalNetworkSendTime: 77.565ms - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 1.19K (1190) - TotalThreadsTotalWallClockTime: 296.133ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 92.000ms - TotalThreadsVoluntaryContextSwitches: 1.30K (1299) Fragment Instance Lifecycle Timings: - ExecTime: 237.536ms - ExecTreeExecTime: 62.175ms - OpenTime: 58.605ms - ExecTreeOpenTime: 58.596ms - PrepareTime: 21.605ms - ExecTreePrepareTime: 13.056us DataStreamSender (dst_id=8):(Total: 159.526ms, non-child: 159.526ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11440692) - NetworkThroughput(*): 145.73 MB/sec - OverallThroughput: 68.39 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 82.260ms - TransmitDataRPCTime: 74.868ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 19.242ms, non-child: 19.242ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 18.943ms EXCHANGE_NODE (id=7):(Total: 120.253ms, non-child: 120.253ms, % non-child: 100.00%) - ConvertRowBatchTime: 30.454ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 16.63 M/sec DataStreamReceiver: - BytesReceived: 14.29 MB (14987650) - DeserializeRowBatchTimer: 23.775ms - FirstBatchArrivalWaitTime: 58.592ms - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 151.835ms - SendersBlockedTotalTimer(*): 367.944ms Averaged Fragment F04:(Total: 569.072ms, non-child: 179.420ms, % non-child: 31.53%) split sizes: min: 413.88 MB, max: 512.00 MB, avg: 479.29 MB, stddev: 46.26 MB completion times: min:537.544ms max:591.467ms mean: 570.402ms stddev:23.540ms execution rates: min:710.89 MB/sec max:952.48 MB/sec mean:843.00 MB/sec stddev:99.92 MB/sec num instances: 3 - AverageThreadTokens: 3.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.08 MB (59854043) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 144.54 MB (151561909) - RowsProduced: 1.97M (1974357) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 181.676ms - TotalStorageWaitTime: 28.318ms - TotalThreadsInvoluntaryContextSwitches: 764 (764) - TotalThreadsTotalWallClockTime: 822.267ms - TotalThreadsSysTime: 1.333ms - TotalThreadsUserTime: 268.000ms - TotalThreadsVoluntaryContextSwitches: 1.90K (1896) Fragment Instance Lifecycle Timings: - ExecTime: 492.295ms - ExecTreeExecTime: 214.398ms - OpenTime: 62.239ms - ExecTreeOpenTime: 28.084us - PrepareTime: 14.520ms - ExecTreePrepareTime: 58.955us DataStreamSender (dst_id=7):(Total: 102.475ms, non-child: 102.475ms, % non-child: 100.00%) - BytesSent: 10.77 MB (11292653) - NetworkThroughput(*): 361.32 MB/sec - OverallThroughput: 105.52 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1.97M (1974357) - SerializeBatchTime: 72.446ms - TransmitDataRPCTime: 29.825ms - UncompressedRowBatchSize: 32.01 MB (33564074) CodeGen:(Total: 73.444ms, non-child: 73.444ms, % non-child: 100.00%) - CodegenTime: 1.432ms - CompileTime: 21.397ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 39.936ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 11.693ms UNION_NODE (id=1):(Total: 213.732ms, non-child: 778.947us, % non-child: 0.36%) - PeakMemoryUsage: 0 - RowsReturned: 1.97M (1974357) - RowsReturnedRate: 9.43 M/sec HDFS_SCAN_NODE (id=3):(Total: 7.984ms, non-child: 7.984ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 21.33 MB (22369621) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 18.68 MB (19592192) - PerReadThreadRawHdfsThroughput: 2.52 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 92.24K (92240) - RowsReturned: 91.82K (91818) - RowsReturnedRate: 3.85 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - DelimiterParseTime: 56.310ms - MaterializeTupleTime(*): 1.685ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 24.829ms - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 204.969ms, non-child: 204.969ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 239.67 MB (251309202) - BytesReadDataNodeCache: 0 - BytesReadLocal: 69.00 MB (72352232) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 69.00 MB (72352232) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 1 (1) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59820032) - PerReadThreadRawHdfsThroughput: 3.20 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 58.582ms - RowBatchQueuePutWaitTime: 76.132ms - RowsRead: 1.88M (1882887) - RowsReturned: 1.88M (1882887) - RowsReturnedRate: 9.39 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 373 (373) - ScannerThreadsTotalWallClockTime: 267.738ms - DelimiterParseTime: 627.482ms - MaterializeTupleTime(*): 61.418ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 133.333ms - ScannerThreadsVoluntaryContextSwitches: 124 (124) - TotalRawHdfsReadTime(*): 75.532ms - TotalReadThroughput: 309.33 MB/sec Fragment F04: Instance 98428cead46af6f9:f8d48d1e00000004 (host=mikeb-desktop:22001):(Total: 591.131ms, non-child: 162.201ms, % non-child: 27.44%) Hdfs split stats (:<# splits>/): 0:4/512.00 MB MemoryUsage(500.000ms): 48.97 MB ThreadUsage(500.000ms): 3 - AverageThreadTokens: 3.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 105.24 MB (110357254) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 180.113ms - TotalStorageWaitTime: 29.966ms - TotalThreadsInvoluntaryContextSwitches: 45 (45) - TotalThreadsTotalWallClockTime: 571.102ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 136.000ms - TotalThreadsVoluntaryContextSwitches: 1.97K (1969) Fragment Instance Lifecycle Timings: - ExecTime: 506.939ms - ExecTreeExecTime: 258.114ms - OpenTime: 64.167ms - ExecTreeOpenTime: 27.770us - PrepareTime: 20.005ms - ExecTreePrepareTime: 60.228us DataStreamSender (dst_id=7):(Total: 94.072ms, non-child: 94.072ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11437550) - NetworkThroughput(*): 371.43 MB/sec - OverallThroughput: 115.95 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 64.531ms - TransmitDataRPCTime: 29.366ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 77.238ms, non-child: 77.238ms, % non-child: 100.00%) - CodegenTime: 1.535ms - CompileTime: 22.666ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 40.678ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 13.458ms UNION_NODE (id=1):(Total: 257.619ms, non-child: 645.899us, % non-child: 0.25%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 7.76 M/sec HDFS_SCAN_NODE (id=3):(Total: 61.290us, non-child: 61.290us, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 256.911ms, non-child: 256.911ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 136.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 256.00 MB (268435456) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 2.65 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 2.00M (2000896) - RowsReturned: 2.00M (2000896) - RowsReturnedRate: 7.79 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - DelimiterParseTime: 809.060ms - MaterializeTupleTime(*): 62.158ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 94.265ms - TotalReadThroughput: 272.00 MB/sec Instance 98428cead46af6f9:f8d48d1e00000006 (host=mikeb-desktop:22002):(Total: 581.846ms, non-child: 194.231ms, % non-child: 33.38%) Hdfs split stats (:<# splits>/): 1:4/413.88 MB MemoryUsage(500.000ms): 48.97 MB ThreadUsage(500.000ms): 3 - AverageThreadTokens: 3.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.07 MB (59841755) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 81.15 MB (85086982) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 184.221ms - TotalStorageWaitTime: 37.478ms - TotalThreadsInvoluntaryContextSwitches: 1.76K (1763) - TotalThreadsTotalWallClockTime: 1s372ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 536.000ms - TotalThreadsVoluntaryContextSwitches: 2.03K (2031) Fragment Instance Lifecycle Timings: - ExecTime: 505.946ms - ExecTreeExecTime: 205.107ms - OpenTime: 63.371ms - ExecTreeOpenTime: 33.142us - PrepareTime: 12.507ms - ExecTreePrepareTime: 66.500us DataStreamSender (dst_id=7):(Total: 108.141ms, non-child: 108.141ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11441297) - NetworkThroughput(*): 369.65 MB/sec - OverallThroughput: 100.90 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 78.451ms - TransmitDataRPCTime: 29.518ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 74.992ms, non-child: 74.992ms, % non-child: 100.00%) - CodegenTime: 1.621ms - CompileTime: 24.204ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 38.366ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 11.962ms UNION_NODE (id=1):(Total: 204.480ms, non-child: 754.313us, % non-child: 0.37%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 9.78 M/sec HDFS_SCAN_NODE (id=3):(Total: 23.845ms, non-child: 23.845ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 64.00 MB (67108864) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.05 MB (58776576) - PerReadThreadRawHdfsThroughput: 2.52 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 276.72K (276722) - RowsReturned: 275.46K (275456) - RowsReturnedRate: 11.55 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - DelimiterParseTime: 56.310ms - MaterializeTupleTime(*): 5.055ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 24.829ms - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 179.880ms, non-child: 179.880ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 BytesRead(500.000ms): 136.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 207.00 MB (217056696) - BytesReadDataNodeCache: 0 - BytesReadLocal: 207.00 MB (217056696) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 207.00 MB (217056696) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 3 (3) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 2 (2) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.04 MB (59807744) - PerReadThreadRawHdfsThroughput: 3.75 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 175.746ms - RowBatchQueuePutWaitTime: 228.397ms - RowsRead: 1.72M (1724693) - RowsReturned: 1.72M (1724693) - RowsReturnedRate: 9.59 M/sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 1.12K (1120) - ScannerThreadsTotalWallClockTime: 803.216ms - DelimiterParseTime: 483.663ms - MaterializeTupleTime(*): 47.330ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 400.000ms - ScannerThreadsVoluntaryContextSwitches: 372 (372) - TotalRawHdfsReadTime(*): 53.974ms - TotalReadThroughput: 272.00 MB/sec Instance 98428cead46af6f9:f8d48d1e00000005 (host=mikeb-desktop:22000):(Total: 534.241ms, non-child: 181.827ms, % non-child: 34.03%) Hdfs split stats (:<# splits>/): 2:4/512.00 MB MemoryUsage(500.000ms): 48.06 MB ThreadUsage(500.000ms): 3 - AverageThreadTokens: 3.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.23 MB (259241492) - RowsProduced: 1.92M (1923072) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 180.695ms - TotalStorageWaitTime: 17.510ms - TotalThreadsInvoluntaryContextSwitches: 484 (484) - TotalThreadsTotalWallClockTime: 523.174ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 132.000ms - TotalThreadsVoluntaryContextSwitches: 1.69K (1688) Fragment Instance Lifecycle Timings: - ExecTime: 463.999ms - ExecTreeExecTime: 179.973ms - OpenTime: 59.178ms - ExecTreeOpenTime: 23.340us - PrepareTime: 11.047ms - ExecTreePrepareTime: 50.138us DataStreamSender (dst_id=7):(Total: 105.213ms, non-child: 105.213ms, % non-child: 100.00%) - BytesSent: 10.49 MB (10999114) - NetworkThroughput(*): 342.87 MB/sec - OverallThroughput: 99.70 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1.92M (1923072) - SerializeBatchTime: 74.354ms - TransmitDataRPCTime: 30.593ms - UncompressedRowBatchSize: 31.18 MB (32692224) CodeGen:(Total: 68.102ms, non-child: 68.102ms, % non-child: 100.00%) - CodegenTime: 1.141ms - CompileTime: 17.323ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 40.765ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 9.659ms UNION_NODE (id=1):(Total: 179.097ms, non-child: 936.628us, % non-child: 0.52%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 1.92M (1923072) - RowsReturnedRate: 10.74 M/sec HDFS_SCAN_NODE (id=3):(Total: 45.554us, non-child: 45.554us, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 178.115ms, non-child: 178.115ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 192.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 256.00 MB (268435456) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 3.19 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.92M (1923072) - RowsReturned: 1.92M (1923072) - RowsReturnedRate: 10.80 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - DelimiterParseTime: 589.722ms - MaterializeTupleTime(*): 74.765ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 78.356ms - TotalReadThroughput: 384.00 MB/sec