Query (id=4c4bc0ba9e38d10b:aa148c4500000000): Summary: Session ID: 5b4aaca21a650825:b6cbe1293df6fcba Session Type: BEESWAX Start Time: 2017-12-21 09:00:07.361445000 End Time: 2017-12-21 09:00:07.954704000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.11.0-RELEASE RELEASE (build Could not obtain git hash) User: mikeb Connected User: mikeb Delegated User: Network Address: ::ffff:127.0.0.1:45245 Default Db: default Sql Statement: with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem) select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a join (select * from l LIMIT 2000000) b on a.l_orderkey = -b.l_orderkey; Coordinator: mikeb-desktop:22000 Query Options (set by configuration): Query Options (set by configuration and planner): MT_DOP=0 Plan: ---------------- Max Per-Host Resource Reservation: Memory=17.00MB Per-Host Resource Estimates: Memory=555.00MB F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=27.00MB mem-reservation=17.00MB PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 05:AGGREGATE [FINALIZE] | output: count(*) | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | tuple-ids=7 row-size=8B cardinality=1 | 04:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: a.l_orderkey = -1 * l_orderkey | fk/pk conjuncts: assumed fk/pk | runtime filters: RF000[bloom] <- -1 * l_orderkey | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB | tuple-ids=0,4 row-size=16B cardinality=1 | |--08:EXCHANGE [UNPARTITIONED] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B | 07:EXCHANGE [UNPARTITIONED] | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F04:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | Per-Host Resources: mem-estimate=264.00MB mem-reservation=0B | 01:UNION | | pass-through-operands: all | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | |--03:SCAN HDFS [tpch.lineitem, RANDOM] | | partitions=1/1 files=1 size=718.94MB | | stored statistics: | | table: rows=6001215 size=718.94MB | | columns: all | | extrapolated-rows=disabled | | mem-estimate=264.00MB mem-reservation=0B | | tuple-ids=3 row-size=8B cardinality=6001215 | | | 02:SCAN HDFS [tpch.lineitem, RANDOM] | partitions=1/1 files=1 size=718.94MB | stored statistics: | table: rows=6001215 size=718.94MB | columns: all | extrapolated-rows=disabled | mem-estimate=264.00MB mem-reservation=0B | tuple-ids=2 row-size=8B cardinality=6001215 | 06:EXCHANGE [UNPARTITIONED] | limit: 1 | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=8B cardinality=1 | F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 Per-Host Resources: mem-estimate=264.00MB mem-reservation=0B 00:SCAN HDFS [tpch.lineitem a, RANDOM] partitions=1/1 files=1 size=718.94MB runtime filters: RF000[bloom] -> a.l_orderkey stored statistics: table: rows=6001215 size=718.94MB columns: all extrapolated-rows=disabled limit: 1 mem-estimate=264.00MB mem-reservation=0B tuple-ids=0 row-size=8B cardinality=1 ---------------- Estimated Per-Host Mem: 581959680 Per Host Min Reservation: mikeb-desktop:22000(17.00 MB) mikeb-desktop:22001(0) mikeb-desktop:22002(0) Request Pool: default-pool Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------- 05:AGGREGATE 1 19.250us 19.250us 1 1 28.00 KB 10.00 MB FINALIZE 04:HASH JOIN 1 1.181ms 1.181ms 0 1 141.06 MB 17.00 MB INNER JOIN, BROADCAST |--08:EXCHANGE 1 30.836ms 30.836ms 2.00M 2.00M 0 0 UNPARTITIONED | 07:EXCHANGE 1 111.858ms 111.858ms 2.00M 2.00M 0 0 UNPARTITIONED | 01:UNION 3 681.451us 765.286us 5.59M 2.00M 0 0 | |--03:SCAN HDFS 3 47.576us 63.608us 0 6.00M 0 264.00 MB tpch.lineitem | 02:SCAN HDFS 3 212.570ms 220.063ms 5.60M 6.00M 57.05 MB 264.00 MB tpch.lineitem 06:EXCHANGE 1 275.356ms 275.356ms 1 1 0 0 UNPARTITIONED 00:SCAN HDFS 3 475.404ms 510.837ms 3 1 56.08 MB 264.00 MB tpch.lineitem a Errors: Planner Timeline: 4.597ms - Analysis finished: 2.066ms (2.066ms) - Value transfer graph computed: 2.159ms (93.008us) - Single node plan created: 3.259ms (1.100ms) - Runtime filters computed: 3.462ms (203.615us) - Distributed plan created: 3.504ms (41.586us) - Planning finished: 4.597ms (1.093ms) Query Timeline: 594.728ms - Query submitted: 17.590us (17.590us) - Planning finished: 5.462ms (5.444ms) - Submit for admission: 5.571ms (108.810us) - Completed admission: 5.593ms (22.250us) - Ready to start on 3 backends: 5.669ms (76.297us) - All 3 execution backends (8 fragment instances) started: 6.228ms (558.480us) - First dynamic filter received: 337.706ms (331.478ms) - Rows available: 490.524ms (152.817ms) - First row fetched: 531.131ms (40.607ms) - Unregister query: 593.262ms (62.130ms) - ComputeScanRangeAssignmentTimer: 24.998us ImpalaServer: - ClientFetchWaitTimer: 43.191ms - RowMaterializationTimer: 59.535ms Execution Profile 4c4bc0ba9e38d10b:aa148c4500000000:(Total: 542.976ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 1 Filter routing table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 0 REMOTE false 1 (1) N/A N/A true Backend startup latencies: Count: 3, min / max: 0 / 0, 25th %-ile: 0, 50th %-ile: 0, 75th %-ile: 0, 90th %-ile: 0, 95th %-ile: 0, 99.9th %-ile: 0 Final filter table: ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled ------------------------------------------------------------------------------------------------------------------- 0 4 0 REMOTE false 0 (1) 337.707ms 337.708ms false Per Node Peak Memory Usage: mikeb-desktop:22000(247.22 MB) mikeb-desktop:22001(105.13 MB) mikeb-desktop:22002(97.15 MB) - FiltersReceived: 1 (1) - FinalizationTimer: 0.000ns - NumBackends: 3 (3) - NumFragmentInstances: 8 (8) - NumFragments: 4 (4) Averaged Fragment F01:(Total: 524.980ms, non-child: 44.656ms, % non-child: 8.51%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:525.372ms max:525.372ms mean: 525.372ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 0.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 142.09 MB (148993814) - PeakReservation: 141.00 MB (147849216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.22 MB (259224084) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 286.085ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 270 (270) - TotalThreadsTotalWallClockTime: 708.496ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 340.000ms - TotalThreadsVoluntaryContextSwitches: 267 (267) Fragment Instance Lifecycle Timings: - ExecTime: 40.634ms - ExecTreeExecTime: 1.882us - OpenTime: 466.715ms - ExecTreeOpenTime: 275.682ms - PrepareTime: 17.597ms - ExecTreePrepareTime: 84.338us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 204.558ms, non-child: 204.558ms, % non-child: 100.00%) - CodegenTime: 172.774us - CompileTime: 45.995ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 87 (87) - NumInstructions: 2.81K (2815) - OptimizationTime: 143.687ms - PeakMemoryUsage: 1.37 MB (1441280) - PrepareTime: 14.434ms AGGREGATION_NODE (id=5):(Total: 275.765ms, non-child: 19.250us, % non-child: 0.01%) - BuildTime: 3.861us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 28.00 KB (28672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 275.746ms, non-child: 1.181ms, % non-child: 0.43%) - BuildRows: 2.00M (2000000) - BuildTime: 168.590ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 141.06 MB (147908864) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.030us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 83.632us - CumulativeAllocationBytes: 141.00 MB (147849216) - CumulativeAllocations: 93 (93) - PeakReservation: 141.00 MB (147849216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 141.00 MB (147849216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=4): - BuildRowsPartitionTime: 88.632ms - BuildRowsPartitioned: 2.00M (2000000) - HashBuckets: 4.19M (4194304) - HashCollisions: 0 (0) - HashTablesBuildTime: 79.092ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=8):(Total: 30.836ms, non-child: 30.836ms, % non-child: 100.00%) - ConvertRowBatchTime: 19.169ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 64.86 M/sec DataStreamReceiver: - BytesReceived: 10.91 MB (11440743) - DeserializeRowBatchTimer: 16.383ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 34.621ms - SendersBlockedTotalTimer(*): 34.622ms EXCHANGE_NODE (id=6):(Total: 275.356ms, non-child: 275.356ms, % non-child: 100.00%) - ConvertRowBatchTime: 1.603us - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec DataStreamReceiver: - BytesReceived: 17.00 B (17) - DeserializeRowBatchTimer: 2.657us - FirstBatchArrivalWaitTime: 275.352ms - PeakMemoryUsage: 4.01 KB (4104) - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Coordinator Fragment F01: Instance 4c4bc0ba9e38d10b:aa148c4500000000 (host=mikeb-desktop:22000):(Total: 524.980ms, non-child: 44.656ms, % non-child: 8.51%) MemoryUsage(500.000ms): 10.05 MB - AverageThreadTokens: 0.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 142.09 MB (148993814) - PeakReservation: 141.00 MB (147849216) - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.22 MB (259224084) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 286.085ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 270 (270) - TotalThreadsTotalWallClockTime: 708.496ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 340.000ms - TotalThreadsVoluntaryContextSwitches: 267 (267) Fragment Instance Lifecycle Timings: - ExecTime: 40.634ms - ExecTreeExecTime: 1.882us - OpenTime: 466.715ms - ExecTreeOpenTime: 275.682ms - PrepareTime: 17.597ms - ExecTreePrepareTime: 84.338us PLAN_ROOT_SINK: - PeakMemoryUsage: 0 CodeGen:(Total: 204.558ms, non-child: 204.558ms, % non-child: 100.00%) - CodegenTime: 172.774us - CompileTime: 45.995ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 87 (87) - NumInstructions: 2.81K (2815) - OptimizationTime: 143.687ms - PeakMemoryUsage: 1.37 MB (1441280) - PrepareTime: 14.434ms AGGREGATION_NODE (id=5):(Total: 275.765ms, non-child: 19.250us, % non-child: 0.01%) ExecOption: Codegen Enabled - BuildTime: 3.861us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 28.00 KB (28672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec - SpilledPartitions: 0 (0) HASH_JOIN_NODE (id=4):(Total: 275.746ms, non-child: 1.181ms, % non-child: 0.43%) ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - BuildRows: 2.00M (2000000) - BuildTime: 168.590ms - NumHashTableBuildsSkipped: 0 (0) - PeakMemoryUsage: 141.06 MB (147908864) - ProbeRows: 1 (1) - ProbeRowsPartitioned: 0 (0) - ProbeTime: 1.030us - RowsReturned: 0 (0) - RowsReturnedRate: 0 Buffer pool: - AllocTime: 83.632us - CumulativeAllocationBytes: 141.00 MB (147849216) - CumulativeAllocations: 93 (93) - PeakReservation: 141.00 MB (147849216) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 141.00 MB (147849216) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns Hash Join Builder (join_node_id=4): ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled Runtime filters: 1 of 1 Runtime Filter Published - BuildRowsPartitionTime: 88.632ms - BuildRowsPartitioned: 2.00M (2000000) - HashBuckets: 4.19M (4194304) - HashCollisions: 0 (0) - HashTablesBuildTime: 79.092ms - LargestPartitionPercent: 6 (6) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 25.12 KB (25728) - RepartitionTime: 0.000ns - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=8):(Total: 30.836ms, non-child: 30.836ms, % non-child: 100.00%) - ConvertRowBatchTime: 19.169ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 64.86 M/sec DataStreamReceiver: BytesReceived(500.000ms): 3.37 MB - BytesReceived: 10.91 MB (11440743) - DeserializeRowBatchTimer: 16.383ms - FirstBatchArrivalWaitTime: 0.000ns - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 34.621ms - SendersBlockedTotalTimer(*): 34.622ms EXCHANGE_NODE (id=6):(Total: 275.356ms, non-child: 275.356ms, % non-child: 100.00%) - ConvertRowBatchTime: 1.603us - PeakMemoryUsage: 0 - RowsReturned: 1 (1) - RowsReturnedRate: 3.00 /sec DataStreamReceiver: BytesReceived(500.000ms): 0 - BytesReceived: 17.00 B (17) - DeserializeRowBatchTimer: 2.657us - FirstBatchArrivalWaitTime: 275.352ms - PeakMemoryUsage: 4.01 KB (4104) - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 533.377ms, non-child: 743.893us, % non-child: 0.14%) split sizes: min: 206.94 MB, max: 256.00 MB, avg: 239.65 MB, stddev: 23.13 MB completion times: min:484.917ms max:560.183ms mean: 533.759ms stddev:34.575ms execution rates: min:369.41 MB/sec max:527.92 MB/sec mean:452.54 MB/sec stddev:64.94 MB/sec num instances: 3 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.09 MB (59860523) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 149.83 MB (157108234) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 169.321us - TotalStorageWaitTime: 21.647ms - TotalThreadsInvoluntaryContextSwitches: 441 (441) - TotalThreadsTotalWallClockTime: 882.851ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 230.666ms - TotalThreadsVoluntaryContextSwitches: 28 (28) Fragment Instance Lifecycle Timings: - ExecTime: 475.525ms - ExecTreeExecTime: 475.304ms - OpenTime: 42.825ms - ExecTreeOpenTime: 23.724us - PrepareTime: 15.002ms - ExecTreePrepareTime: 34.905us DataStreamSender (dst_id=6):(Total: 807.647us, non-child: 807.647us, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 1.37 MB/sec - OverallThroughput: 127.79 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 2.627us - TransmitDataRPCTime: 12.010us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 56.420ms, non-child: 56.420ms, % non-child: 100.00%) - CodegenTime: 1.254ms - CompileTime: 16.100ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 25.941ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 13.949ms HDFS_SCAN_NODE (id=0):(Total: 475.404ms, non-child: 475.404ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 141.33 MB (148198741) - BytesReadDataNodeCache: 0 - BytesReadLocal: 141.33 MB (148198741) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 141.33 MB (148198741) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.06 MB (58787840) - PerReadThreadRawHdfsThroughput: 2.64 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 170.500ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 874.08K (874077) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 409 (409) - ScannerThreadsTotalWallClockTime: 364.509ms - DelimiterParseTime: 317.332ms - MaterializeTupleTime(*): 262.867us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 188.000ms - ScannerThreadsVoluntaryContextSwitches: 8 (8) - TotalRawHdfsReadTime(*): 54.058ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Fragment F00: Instance 4c4bc0ba9e38d10b:aa148c4500000001 (host=mikeb-desktop:22002):(Total: 559.858ms, non-child: 589.253us, % non-child: 0.11%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB Filter 0 arrival: 335ms MemoryUsage(500.000ms): 23.54 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.10 MB (59872811) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 97.15 MB (101865222) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 40.872us - TotalStorageWaitTime: 16.272ms - TotalThreadsInvoluntaryContextSwitches: 385 (385) - TotalThreadsTotalWallClockTime: 958.762ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 268.000ms - TotalThreadsVoluntaryContextSwitches: 27 (27) Fragment Instance Lifecycle Timings: - ExecTime: 497.946ms - ExecTreeExecTime: 497.868ms - OpenTime: 47.738ms - ExecTreeOpenTime: 24.744us - PrepareTime: 14.155ms - ExecTreePrepareTime: 28.337us DataStreamSender (dst_id=6):(Total: 266.318us, non-child: 266.318us, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 1.64 MB/sec - OverallThroughput: 62.34 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 2.613us - TransmitDataRPCTime: 9.895us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 61.044ms, non-child: 61.044ms, % non-child: 100.00%) - CodegenTime: 1.155ms - CompileTime: 18.657ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 28.470ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 13.513ms HDFS_SCAN_NODE (id=0):(Total: 497.957ms, non-child: 497.957ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: All filters arrived. Waited 283ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 152.00 MB (159383552) - BytesReadDataNodeCache: 0 - BytesReadLocal: 152.00 MB (159383552) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 152.00 MB (159383552) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.08 MB (58800128) - PerReadThreadRawHdfsThroughput: 3.14 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 206.234ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 974.42K (974423) - RowsReturned: 1 (1) - RowsReturnedRate: 2.00 /sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 340 (340) - ScannerThreadsTotalWallClockTime: 413.086ms - DelimiterParseTime: 395.242ms - MaterializeTupleTime(*): 304.817us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 220.000ms - ScannerThreadsVoluntaryContextSwitches: 6 (6) - TotalRawHdfsReadTime(*): 47.348ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Instance 4c4bc0ba9e38d10b:aa148c4500000003 (host=mikeb-desktop:22001):(Total: 555.861ms, non-child: 743.333us, % non-child: 0.13%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Filter 0 arrival: 334ms MemoryUsage(500.000ms): 23.54 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.08 MB (59854379) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 105.13 MB (110235398) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 98.471us - TotalStorageWaitTime: 30.558ms - TotalThreadsInvoluntaryContextSwitches: 758 (758) - TotalThreadsTotalWallClockTime: 921.092ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 228.000ms - TotalThreadsVoluntaryContextSwitches: 33 (33) Fragment Instance Lifecycle Timings: - ExecTime: 510.882ms - ExecTreeExecTime: 510.742ms - OpenTime: 31.202ms - ExecTreeOpenTime: 20.318us - PrepareTime: 13.757ms - ExecTreePrepareTime: 31.947us DataStreamSender (dst_id=6):(Total: 53.017us, non-child: 53.017us, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 1.27 MB/sec - OverallThroughput: 313.14 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 3.020us - TransmitDataRPCTime: 12.764us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 44.227ms, non-child: 44.227ms, % non-child: 100.00%) - CodegenTime: 1.267ms - CompileTime: 11.250ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 19.352ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 13.207ms HDFS_SCAN_NODE (id=0):(Total: 510.837ms, non-child: 510.837ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: All filters arrived. Waited 301ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 152.00 MB (159383552) - BytesReadDataNodeCache: 0 - BytesReadLocal: 152.00 MB (159383552) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 152.00 MB (159383552) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.06 MB (58781696) - PerReadThreadRawHdfsThroughput: 2.07 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 173.131ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 960.50K (960502) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 752 (752) - ScannerThreadsTotalWallClockTime: 379.016ms - DelimiterParseTime: 310.757ms - MaterializeTupleTime(*): 285.278us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 196.000ms - ScannerThreadsVoluntaryContextSwitches: 12 (12) - TotalRawHdfsReadTime(*): 71.825ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Instance 4c4bc0ba9e38d10b:aa148c4500000002 (host=mikeb-desktop:22000):(Total: 484.411ms, non-child: 899.095us, % non-child: 0.19%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB Filter 0 arrival: 333ms MemoryUsage(500.000ms): 23.54 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 1.00 MB (1048576) - PeakMemoryUsage: 57.08 MB (59854379) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.22 MB (259224084) - RowsProduced: 1 (1) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 368.620us - TotalStorageWaitTime: 18.110ms - TotalThreadsInvoluntaryContextSwitches: 181 (181) - TotalThreadsTotalWallClockTime: 768.698ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 196.000ms - TotalThreadsVoluntaryContextSwitches: 24 (24) Fragment Instance Lifecycle Timings: - ExecTime: 417.747ms - ExecTreeExecTime: 417.302ms - OpenTime: 49.535ms - ExecTreeOpenTime: 26.112us - PrepareTime: 17.094ms - ExecTreePrepareTime: 44.432us DataStreamSender (dst_id=6):(Total: 2.103ms, non-child: 2.103ms, % non-child: 100.00%) - BytesSent: 17.00 B (17) - NetworkThroughput(*): 1.21 MB/sec - OverallThroughput: 7.89 KB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1 (1) - SerializeBatchTime: 2.250us - TransmitDataRPCTime: 13.372us - UncompressedRowBatchSize: 17.00 B (17) CodeGen:(Total: 63.990ms, non-child: 63.990ms, % non-child: 100.00%) - CodegenTime: 1.339ms - CompileTime: 18.392ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 27 (27) - NumInstructions: 416 (416) - OptimizationTime: 30.001ms - PeakMemoryUsage: 208.00 KB (212992) - PrepareTime: 15.129ms HDFS_SCAN_NODE (id=0):(Total: 417.418ms, non-child: 417.418ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB ExecOption: TEXT Codegen Enabled, Codegen enabled: 2 out of 2 Runtime filters: All filters arrived. Waited 285ms Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 120.00 MB (125829120) - BytesReadDataNodeCache: 0 - BytesReadLocal: 120.00 MB (125829120) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 120.00 MB (125829120) - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 2 (2) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 (1) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 56.06 MB (58781696) - PerReadThreadRawHdfsThroughput: 2.73 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 132.134ms - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 687.31K (687306) - RowsReturned: 1 (1) - RowsReturnedRate: 2.00 /sec - ScanRangesComplete: 2 (2) - ScannerThreadsInvoluntaryContextSwitches: 136 (136) - ScannerThreadsTotalWallClockTime: 301.424ms - DelimiterParseTime: 245.998ms - MaterializeTupleTime(*): 198.507us - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 148.000ms - ScannerThreadsVoluntaryContextSwitches: 7 (7) - TotalRawHdfsReadTime(*): 43.002ms - TotalReadThroughput: 0.00 /sec Filter 0 (1.00 MB): - Files processed: 0 (0) - Files rejected: 0 (0) - Files total: 0 (0) - RowGroups processed: 0 (0) - RowGroups rejected: 0 (0) - RowGroups total: 0 (0) - Rows processed: 0 (0) - Rows rejected: 0 (0) - Rows total: 0 (0) - Splits processed: 0 (0) - Splits rejected: 0 (0) - Splits total: 0 (0) Averaged Fragment F05:(Total: 329.865ms, non-child: 27.848ms, % non-child: 8.44%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:330.549ms max:330.549ms mean: 330.549ms stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 10.04 MB (10528992) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 93.53 MB (98071284) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 85.063ms - TotalNetworkSendTime: 124.500ms - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 594 (594) - TotalThreadsTotalWallClockTime: 316.515ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 100.000ms - TotalThreadsVoluntaryContextSwitches: 1.31K (1309) Fragment Instance Lifecycle Timings: - ExecTime: 252.562ms - ExecTreeExecTime: 48.487ms - OpenTime: 63.958ms - ExecTreeOpenTime: 63.951ms - PrepareTime: 13.323ms - ExecTreePrepareTime: 16.018us DataStreamSender (dst_id=8):(Total: 176.996ms, non-child: 176.996ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11440743) - NetworkThroughput(*): 105.32 MB/sec - OverallThroughput: 61.64 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 73.195ms - TransmitDataRPCTime: 103.599ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 13.160ms, non-child: 13.160ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 12.725ms EXCHANGE_NODE (id=7):(Total: 111.858ms, non-child: 111.858ms, % non-child: 100.00%) - ConvertRowBatchTime: 25.764ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 17.88 M/sec DataStreamReceiver: - BytesReceived: 14.29 MB (14988921) - DeserializeRowBatchTimer: 23.793ms - FirstBatchArrivalWaitTime: 63.947ms - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 151.535ms - SendersBlockedTotalTimer(*): 385.588ms Fragment F05: Instance 4c4bc0ba9e38d10b:aa148c4500000007 (host=mikeb-desktop:22000):(Total: 329.865ms, non-child: 27.848ms, % non-child: 8.44%) MemoryUsage(500.000ms): 10.04 MB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 10.04 MB (10528992) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 93.53 MB (98071284) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 85.063ms - TotalNetworkSendTime: 124.500ms - TotalStorageWaitTime: 0.000ns - TotalThreadsInvoluntaryContextSwitches: 594 (594) - TotalThreadsTotalWallClockTime: 316.515ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 100.000ms - TotalThreadsVoluntaryContextSwitches: 1.31K (1309) Fragment Instance Lifecycle Timings: - ExecTime: 252.562ms - ExecTreeExecTime: 48.487ms - OpenTime: 63.958ms - ExecTreeOpenTime: 63.951ms - PrepareTime: 13.323ms - ExecTreePrepareTime: 16.018us DataStreamSender (dst_id=8):(Total: 176.996ms, non-child: 176.996ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11440743) - NetworkThroughput(*): 105.32 MB/sec - OverallThroughput: 61.64 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 73.195ms - TransmitDataRPCTime: 103.599ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 13.160ms, non-child: 13.160ms, % non-child: 100.00%) - CodegenTime: 0.000ns - CompileTime: 0.000ns - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 0 (0) - NumInstructions: 0 (0) - OptimizationTime: 0.000ns - PeakMemoryUsage: 0 - PrepareTime: 12.725ms EXCHANGE_NODE (id=7):(Total: 111.858ms, non-child: 111.858ms, % non-child: 100.00%) - ConvertRowBatchTime: 25.764ms - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 17.88 M/sec DataStreamReceiver: BytesReceived(500.000ms): 6.75 MB - BytesReceived: 14.29 MB (14988921) - DeserializeRowBatchTimer: 23.793ms - FirstBatchArrivalWaitTime: 63.947ms - PeakMemoryUsage: 10.01 MB (10497024) - SendersBlockedTimer: 151.535ms - SendersBlockedTotalTimer(*): 385.588ms Averaged Fragment F04:(Total: 561.360ms, non-child: 181.721ms, % non-child: 32.37%) split sizes: min: 413.88 MB, max: 512.00 MB, avg: 479.29 MB, stddev: 46.26 MB completion times: min:544.957ms max:583.859ms mean: 563.155ms stddev:15.980ms execution rates: min:738.21 MB/sec max:939.52 MB/sec mean:851.55 MB/sec stddev:84.12 MB/sec num instances: 3 - AverageThreadTokens: 1.67 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 149.83 MB (157108234) - RowsProduced: 1.86M (1864977) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 200.726ms - TotalStorageWaitTime: 29.036ms - TotalThreadsInvoluntaryContextSwitches: 702 (702) - TotalThreadsTotalWallClockTime: 845.020ms - TotalThreadsSysTime: 6.666ms - TotalThreadsUserTime: 277.333ms - TotalThreadsVoluntaryContextSwitches: 1.84K (1841) Fragment Instance Lifecycle Timings: - ExecTime: 483.245ms - ExecTreeExecTime: 213.767ms - OpenTime: 63.491ms - ExecTreeOpenTime: 27.537us - PrepareTime: 14.607ms - ExecTreePrepareTime: 59.693us DataStreamSender (dst_id=7):(Total: 90.619ms, non-child: 90.619ms, % non-child: 100.00%) - BytesSent: 10.17 MB (10666928) - NetworkThroughput(*): 347.94 MB/sec - OverallThroughput: 112.12 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1.86M (1864977) - SerializeBatchTime: 60.853ms - TransmitDataRPCTime: 29.556ms - UncompressedRowBatchSize: 30.24 MB (31704614) CodeGen:(Total: 75.719ms, non-child: 75.719ms, % non-child: 100.00%) - CodegenTime: 1.592ms - CompileTime: 21.702ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 40.845ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 12.738ms UNION_NODE (id=1):(Total: 213.299ms, non-child: 681.452us, % non-child: 0.32%) - PeakMemoryUsage: 0 - RowsReturned: 1.86M (1864977) - RowsReturnedRate: 8.79 M/sec HDFS_SCAN_NODE (id=3):(Total: 47.576us, non-child: 47.576us, % non-child: 100.00%) - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 212.570ms, non-child: 212.570ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.67 - BytesRead: 239.67 MB (251309202) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 3.19 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.87M (1865708) - RowsReturned: 1.87M (1865557) - RowsReturnedRate: 8.82 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 566 (566) - ScannerThreadsTotalWallClockTime: 298.290ms - DelimiterParseTime: 598.675ms - MaterializeTupleTime(*): 57.519ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 149.333ms - ScannerThreadsVoluntaryContextSwitches: 56 (56) - TotalRawHdfsReadTime(*): 79.928ms - TotalReadThroughput: 64.00 MB/sec Fragment F04: Instance 4c4bc0ba9e38d10b:aa148c4500000005 (host=mikeb-desktop:22000):(Total: 579.305ms, non-child: 181.906ms, % non-child: 31.40%) Hdfs split stats (:<# splits>/): 2:4/512.00 MB MemoryUsage(500.000ms): 49.07 MB ThreadUsage(500.000ms): 3 - AverageThreadTokens: 3.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 247.22 MB (259224084) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 207.823ms - TotalStorageWaitTime: 17.489ms - TotalThreadsInvoluntaryContextSwitches: 1.10K (1098) - TotalThreadsTotalWallClockTime: 1s054ms - TotalThreadsSysTime: 4.000ms - TotalThreadsUserTime: 384.000ms - TotalThreadsVoluntaryContextSwitches: 1.99K (1991) Fragment Instance Lifecycle Timings: - ExecTime: 499.685ms - ExecTreeExecTime: 221.234ms - OpenTime: 65.775ms - ExecTreeOpenTime: 30.943us - PrepareTime: 13.825ms - ExecTreePrepareTime: 89.644us DataStreamSender (dst_id=7):(Total: 97.944ms, non-child: 97.944ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11439602) - NetworkThroughput(*): 307.32 MB/sec - OverallThroughput: 111.39 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 62.240ms - TransmitDataRPCTime: 35.499ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 78.683ms, non-child: 78.683ms, % non-child: 100.00%) - CodegenTime: 1.582ms - CompileTime: 23.622ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 41.065ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 13.573ms UNION_NODE (id=1):(Total: 220.771ms, non-child: 644.357us, % non-child: 0.29%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 9.06 M/sec HDFS_SCAN_NODE (id=3):(Total: 63.608us, non-child: 63.608us, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 220.063ms, non-child: 220.063ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 2:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 96.00 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 2.00 - BytesRead: 256.06 MB (268500992) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 3.48 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 2.00M (2000844) - RowsReturned: 2.00M (2000844) - RowsReturnedRate: 9.09 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 959 (959) - ScannerThreadsTotalWallClockTime: 489.156ms - DelimiterParseTime: 589.377ms - MaterializeTupleTime(*): 73.763ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 248.000ms - ScannerThreadsVoluntaryContextSwitches: 98 (98) - TotalRawHdfsReadTime(*): 71.818ms - TotalReadThroughput: 192.00 MB/sec Instance 4c4bc0ba9e38d10b:aa148c4500000006 (host=mikeb-desktop:22002):(Total: 560.208ms, non-child: 188.561ms, % non-child: 33.66%) Hdfs split stats (:<# splits>/): 1:4/413.88 MB MemoryUsage(500.000ms): 22.22 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 97.15 MB (101865222) - RowsProduced: 1.59M (1594932) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 198.054ms - TotalStorageWaitTime: 26.987ms - TotalThreadsInvoluntaryContextSwitches: 936 (936) - TotalThreadsTotalWallClockTime: 949.925ms - TotalThreadsSysTime: 16.000ms - TotalThreadsUserTime: 304.000ms - TotalThreadsVoluntaryContextSwitches: 1.54K (1545) Fragment Instance Lifecycle Timings: - ExecTime: 485.939ms - ExecTreeExecTime: 221.246ms - OpenTime: 58.277ms - ExecTreeOpenTime: 21.831us - PrepareTime: 15.977ms - ExecTreePrepareTime: 40.155us DataStreamSender (dst_id=7):(Total: 79.583ms, non-child: 79.583ms, % non-child: 100.00%) - BytesSent: 8.70 MB (9123940) - NetworkThroughput(*): 362.60 MB/sec - OverallThroughput: 109.34 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 1.59M (1594932) - SerializeBatchTime: 55.375ms - TransmitDataRPCTime: 23.997ms - UncompressedRowBatchSize: 25.86 MB (27113844) CodeGen:(Total: 71.262ms, non-child: 71.262ms, % non-child: 100.00%) - CodegenTime: 1.561ms - CompileTime: 17.291ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 40.195ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 13.347ms UNION_NODE (id=1):(Total: 220.802ms, non-child: 765.286us, % non-child: 0.35%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 1.59M (1594932) - RowsReturnedRate: 7.22 M/sec HDFS_SCAN_NODE (id=3):(Total: 37.865us, non-child: 37.865us, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 219.999ms, non-child: 219.999ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 1:2/206.94 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 206.94 MB (216991160) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 3.94 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 1.59M (1594932) - RowsReturned: 1.59M (1594932) - RowsReturnedRate: 7.25 M/sec - ScanRangesComplete: 1 (1) - ScannerThreadsInvoluntaryContextSwitches: 741 (741) - ScannerThreadsTotalWallClockTime: 405.713ms - DelimiterParseTime: 619.416ms - MaterializeTupleTime(*): 44.481ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 200.000ms - ScannerThreadsVoluntaryContextSwitches: 70 (70) - TotalRawHdfsReadTime(*): 51.227ms - TotalReadThroughput: 0.00 /sec Instance 4c4bc0ba9e38d10b:aa148c4500000004 (host=mikeb-desktop:22001):(Total: 544.566ms, non-child: 174.696ms, % non-child: 32.08%) Hdfs split stats (:<# splits>/): 0:4/512.00 MB MemoryUsage(500.000ms): 22.22 KB ThreadUsage(500.000ms): 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 57.09 MB (59860187) - PeakReservation: 0 - PeakUsedReservation: 0 - PerHostPeakMemUsage: 105.13 MB (110235398) - RowsProduced: 2.00M (2000000) - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 196.301ms - TotalStorageWaitTime: 42.632ms - TotalThreadsInvoluntaryContextSwitches: 73 (73) - TotalThreadsTotalWallClockTime: 530.526ms - TotalThreadsSysTime: 0.000ns - TotalThreadsUserTime: 144.000ms - TotalThreadsVoluntaryContextSwitches: 1.99K (1987) Fragment Instance Lifecycle Timings: - ExecTime: 464.111ms - ExecTreeExecTime: 198.820ms - OpenTime: 66.420ms - ExecTreeOpenTime: 29.838us - PrepareTime: 14.018ms - ExecTreePrepareTime: 49.282us DataStreamSender (dst_id=7):(Total: 94.331ms, non-child: 94.331ms, % non-child: 100.00%) - BytesSent: 10.91 MB (11437242) - NetworkThroughput(*): 373.90 MB/sec - OverallThroughput: 115.63 MB/sec - PeakMemoryUsage: 14.22 KB (14560) - RowsReturned: 2.00M (2000000) - SerializeBatchTime: 64.944ms - TransmitDataRPCTime: 29.172ms - UncompressedRowBatchSize: 32.42 MB (34000000) CodeGen:(Total: 77.214ms, non-child: 77.214ms, % non-child: 100.00%) - CodegenTime: 1.634ms - CompileTime: 24.193ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.89 MB (1980104) - NumFunctions: 32 (32) - NumInstructions: 557 (557) - OptimizationTime: 41.275ms - PeakMemoryUsage: 278.50 KB (285184) - PrepareTime: 11.294ms UNION_NODE (id=1):(Total: 198.324ms, non-child: 634.712us, % non-child: 0.32%) ExecOption: Codegen Enabled - PeakMemoryUsage: 0 - RowsReturned: 2.00M (2000000) - RowsReturnedRate: 10.08 M/sec HDFS_SCAN_NODE (id=3):(Total: 41.256us, non-child: 41.256us, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - BytesRead: 0 - CollectionItemsRead: 0 (0) - PeakMemoryUsage: 0 - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - MaterializeTupleTime(*): 0.000ns - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalReadThroughput: 0.00 /sec HDFS_SCAN_NODE (id=2):(Total: 197.648ms, non-child: 197.648ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:2/256.00 MB ExecOption: TEXT Codegen Enabled BytesRead(500.000ms): 0 - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 0.00 - BytesRead: 256.00 MB (268435456) - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - CachedFileHandlesHitCount: 0 (0) - CachedFileHandlesMissCount: 0 (0) - CollectionItemsRead: 0 (0) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 0 (0) - NumScannerThreadsStarted: 2 (2) - PeakMemoryUsage: 57.05 MB (59826176) - PerReadThreadRawHdfsThroughput: 2.14 GB/sec - RemoteScanRanges: 0 (0) - RowBatchQueueGetWaitTime: 0.000ns - RowBatchQueuePutWaitTime: 0.000ns - RowsRead: 2.00M (2001350) - RowsReturned: 2.00M (2000896) - RowsReturnedRate: 10.12 M/sec - ScanRangesComplete: 0 (0) - ScannerThreadsInvoluntaryContextSwitches: 0 (0) - ScannerThreadsTotalWallClockTime: 0.000ns - DelimiterParseTime: 587.230ms - MaterializeTupleTime(*): 54.312ms - ScannerThreadsSysTime: 0.000ns - ScannerThreadsUserTime: 0.000ns - ScannerThreadsVoluntaryContextSwitches: 0 (0) - TotalRawHdfsReadTime(*): 116.740ms - TotalReadThroughput: 0.00 /sec