Query (id=d64b2ee4abd82c78:814dfd697b0710bc): Summary: Session ID: a0432f0879fbdee5:8e020a1ac459b99f Session Type: BEESWAX Start Time: 2016-12-30 09:25:44.824510000 End Time: 2016-12-30 09:26:14.431191000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.6.0-cdh5.8.3 RELEASE (build c644f476b774db9db87a619628f7a6ecc5f843e0) User: dejanp Connected User: dejanp Delegated User: Network Address: 88.150.218.17:54802 Default Db: dejanp Sql Statement: with battles AS ( SELECT * FROM event e, (SELECT MAX(CASE WHEN key='won' THEN value END) AS won FROM e.event_map) m WHERE e.date_id = 2420 AND e.event_id = 10012 ) SELECT won, count(*) FROM battles GROUP BY won Coordinator: datanode-02.zamboni.test.hadoop.bol.nordeus.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=882.00MB VCores=2 WARNING: The following tables are missing relevant table and/or column statistics. dejanp.event 09:EXCHANGE [UNPARTITIONED] | hosts=3 per-host-mem=unavailable | tuple-ids=5 row-size=24B cardinality=26485020 | 08:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: won | hosts=3 per-host-mem=128.00MB | tuple-ids=5 row-size=24B cardinality=26485020 | 07:EXCHANGE [HASH(won)] | hosts=3 per-host-mem=0B | tuple-ids=5 row-size=24B cardinality=26485020 | 06:AGGREGATE [STREAMING] | output: count(*) | group by: max(CASE WHEN key = 'won' THEN value END) | hosts=3 per-host-mem=128.00MB | tuple-ids=5 row-size=24B cardinality=26485020 | 01:SUBPLAN | hosts=3 per-host-mem=0B | tuple-ids=2,0 row-size=36B cardinality=26485020 | |--05:NESTED LOOP JOIN [CROSS JOIN] | | hosts=3 per-host-mem=20B | | tuple-ids=2,0 row-size=36B cardinality=1 | | | |--02:SINGULAR ROW SRC | | parent-subplan=01 | | hosts=3 per-host-mem=0B | | tuple-ids=0 row-size=20B cardinality=1 | | | 04:AGGREGATE [FINALIZE] | | output: max(CASE WHEN key = 'won' THEN value END) | | hosts=3 per-host-mem=10.00MB | | tuple-ids=2 row-size=16B cardinality=1 | | | 03:UNNEST [e.event_map] | parent-subplan=01 | hosts=3 per-host-mem=0B | tuple-ids=1 row-size=0B cardinality=10 | 00:SCAN HDFS [dejanp.event e, RANDOM] partitions=1/1 files=64 size=18.76GB predicates: e.event_id = 10012 table stats: unavailable columns missing stats: event_map, user_id, event_id, event_type_id, ts_bigint, processed_ts_bigint, server_id hosts=3 per-host-mem=616.00MB tuple-ids=0 row-size=20B cardinality=26485020 ---------------- Estimated Per-Host Mem: 924844052 Estimated Per-Host VCores: 2 Tables Missing Stats: dejanp.event Request Pool: root.dejanp ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------- 09:EXCHANGE 1 195.396us 195.396us 833 26.49M 0 -1.00 B UNPARTITIONED 08:AGGREGATE 3 179.400ms 183.008ms 833 26.49M 2.29 MB 128.00 MB FINALIZE 07:EXCHANGE 3 38.025us 49.712us 1.00K 26.49M 0 0 HASH(won) 06:AGGREGATE 3 8s990ms 9s515ms 1.00K 26.49M 41.58 MB 128.00 MB STREAMING 01:SUBPLAN 3 1s732ms 1s796ms 15.57M 26.49M 27.62 MB 0 |--05:NESTED LOOP JOIN 3 8s483ms 8s978ms 0 1 1.02 MB 20.00 B CROSS JOIN | |--02:SINGULAR ROW SRC 3 0.000ns 0.000ns 0 1 0 0 | 04:AGGREGATE 3 7s561ms 7s785ms 0 1 2.02 MB 10.00 MB FINALIZE | 03:UNNEST 3 1s091ms 1s136ms 0 10 0 0 e.event_map 00:SCAN HDFS 3 87.031ms 97.391ms 20.06M 26.49M 1.83 GB 616.00 MB dejanp.event e Planner Timeline: 8.276ms - Analysis finished: 4.394ms (4.394ms) - Equivalence classes computed: 4.589ms (195.121us) - Single node plan created: 6.331ms (1.741ms) - Runtime filters computed: 6.338ms (7.298us) - Distributed plan created: 7.079ms (740.685us) - Lineage info computed: 7.256ms (177.133us) - Planning finished: 8.276ms (1.020ms) Query Timeline: 29s617ms - Start execution: 216.068us (216.068us) - Planning finished: 10.924ms (10.708ms) - Ready to start 6 remote fragments: 14.781ms (3.856ms) - All 6 remote fragments started: 17.074ms (2.292ms) - Rows available: 29s400ms (29s383ms) - First row fetched: 29s599ms (199.011ms) - Unregister query: 29s606ms (7.415ms) ImpalaServer: - ClientFetchWaitTimer: 203.183ms - RowMaterializationTimer: 1.722ms Execution Profile d64b2ee4abd82c78:814dfd697b0710bc:(Total: 29s388ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 6, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: datanode-03.zamboni.test.hadoop.bol.nordeus.com:22000(1.86 GB) datanode-01.zamboni.test.hadoop.bol.nordeus.com:22000(1.86 GB) datanode-02.zamboni.test.hadoop.bol.nordeus.com:22000(1.48 GB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F02:(Total: 29s383ms, non-child: 1.172ms, % non-child: 0.00%) MemoryUsage(500.000ms): 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 200.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 206.51 KB (211464) - PerHostPeakMemUsage: 0 - PrepareTime: 217.032us - RowsProduced: 833 (833) - TotalCpuTime: 204.047ms - TotalNetworkReceiveTime: 29s382ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 48 (48) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 48.00 GB (51539607552) - PeakMemoryUsage: 784.00 KB (802816) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns EXCHANGE_NODE (id=9):(Total: 29s382ms, non-child: 195.396us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17.01 KB - BytesReceived: 17.01 KB (17416) - ConvertRowBatchTime: 43.316us - DeserializeRowBatchTimer: 1.917ms - FirstBatchArrivalWaitTime: 29s382ms - PeakMemoryUsage: 0 - RowsReturned: 833 (833) - RowsReturnedRate: 28.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F01:(Total: 29s309ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:29s390ms max:29s393ms mean: 29s391ms stddev:1.398ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 2.30 MB (2415640) - PerHostPeakMemUsage: 1.73 GB (1861008752) - PrepareTime: 178.441ms - RowsProduced: 277 (277) - TotalCpuTime: 261.476ms - TotalNetworkReceiveTime: 29s124ms - TotalNetworkSendTime: 4.471ms - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 48 (48) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 48.00 GB (51539607552) - PeakMemoryUsage: 784.00 KB (802816) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 258.021ms, non-child: 258.021ms, % non-child: 100.00%) - CodegenTime: 821.977us - CompileTime: 15.976ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 42 (42) - NumInstructions: 852 (852) - OptimizationTime: 64.649ms - PrepareTime: 176.911ms DataStreamSender (dst_id=9):(Total: 645.652us, non-child: 645.652us, % non-child: 100.00%) - BytesSent: 5.67 KB (5805) - NetworkThroughput(*): 13.45 MB/sec - OverallThroughput: 8.64 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - RowsReturned: 277 (277) - SerializeBatchTime: 213.048us - TransmitDataRPCTime: 417.396us - UncompressedRowBatchSize: 10.91 KB (11170) AGGREGATION_NODE (id=8):(Total: 29s303ms, non-child: 179.400ms, % non-child: 0.61%) - BuildTime: 74.517us - GetNewBlockTime: 71.961us - GetResultsTime: 78.972us - HTResizeTime: 2.457us - HashBuckets: 16.38K (16384) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.29 MB (2401570) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 277 (277) - RowsReturnedRate: 9.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 473.000ns EXCHANGE_NODE (id=7):(Total: 29s124ms, non-child: 29s124ms, % non-child: 100.00%) - BytesReceived: 5.79 KB (5931) - ConvertRowBatchTime: 11.017us - DeserializeRowBatchTimer: 172.784us - FirstBatchArrivalWaitTime: 26s647ms - PeakMemoryUsage: 0 - RowsReturned: 333 (333) - RowsReturnedRate: 11.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 27s950ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 6.21 GB, max: 6.29 GB, avg: 6.25 GB, stddev: 34.29 MB completion times: min:26s909ms max:29s386ms mean: 28s038ms stddev:1s023ms execution rates: min:218.00 MB/sec max:239.44 MB/sec mean:228.68 MB/sec stddev:8.75 MB/sec num instances: 3 - AverageThreadTokens: 19.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.73 GB (1858612805) - PerHostPeakMemUsage: 1.73 GB (1861008752) - PrepareTime: 215.364ms - RowsProduced: 333 (333) - TotalCpuTime: 8m36s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.340ms - TotalStorageWaitTime: 24s841ms CodeGen:(Total: 268.488ms, non-child: 268.488ms, % non-child: 100.00%) - CodegenTime: 62.778ms - CompileTime: 17.845ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 52 (52) - NumInstructions: 1.11K (1106) - OptimizationTime: 67.858ms - PrepareTime: 182.338ms DataStreamSender (dst_id=7):(Total: 238.097us, non-child: 238.097us, % non-child: 100.00%) - BytesSent: 5.79 KB (5931) - NetworkThroughput(*): 46.20 MB/sec - OverallThroughput: 23.78 MB/sec - PeakMemoryUsage: 48.00 KB (49152) - RowsReturned: 333 (333) - SerializeBatchTime: 98.792us - TransmitDataRPCTime: 124.560us - UncompressedRowBatchSize: 13.03 KB (13345) AGGREGATION_NODE (id=6):(Total: 27s947ms, non-child: 8s990ms, % non-child: 32.17%) - GetNewBlockTime: 36.594us - GetResultsTime: 58.584us - HTResizeTime: 297.668us - HashBuckets: 32.77K (32768) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 40.61 MB (42586045) - PinTime: 0.000ns - ReductionFactorEstimate: 0.00 - ReductionFactorThresholdToExpand: 0.00 - RowsPassedThrough: 0 (0) - RowsReturned: 333 (333) - RowsReturnedRate: 11.00 /sec - StreamingTime: 8s787ms - UnpinTime: 0.000ns SUBPLAN_NODE (id=1):(Total: 18s957ms, non-child: 1s732ms, % non-child: 9.14%) - PeakMemoryUsage: 25.96 MB (27218912) - RowsReturned: 5.19M (5190446) - RowsReturnedRate: 273.86 K/sec NESTED_LOOP_JOIN_NODE (id=5):(Total: 17s137ms, non-child: 8s483ms, % non-child: 49.50%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 1.02 MB (1064960) - ProbeRows: 6.69M (6687619) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 AGGREGATION_NODE (id=4):(Total: 8s653ms, non-child: 7s561ms, % non-child: 87.38%) - BuildTime: 2s688ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 2.02 MB (2113536) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) UNNEST_NODE (id=3):(Total: 1s091ms, non-child: 1s091ms, % non-child: 100.00%) - AvgCollectionSize: 3.00 - MaxCollectionSize: 3 (3) - MinCollectionSize: 3 (3) - NumCollections: 6.69M (6687619) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 87.031ms, non-child: 87.031ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.93 - AverageScannerThreadConcurrency: 18.20 - BytesRead: 5.28 GB (5671112327) - BytesReadDataNodeCache: 0 - BytesReadLocal: 5.28 GB (5671112327) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5.28 GB (5671112327) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 3 (3) - NumRowGroups: 63 (63) - NumScannerThreadsStarted: 20 (20) - PeakMemoryUsage: 1.70 GB (1830717077) - PerReadThreadRawHdfsThroughput: 294.96 MB/sec - RemoteScanRanges: 0 (0) - RowsRead: 575.34M (575340108) - RowsReturned: 6.69M (6687619) - RowsReturnedRate: 77.55 M/sec - ScanRangesComplete: 63 (63) - ScannerThreadsInvoluntaryContextSwitches: 476 (476) - ScannerThreadsTotalWallClockTime: 8m27s - MaterializeTupleTime(*): 7m58s - ScannerThreadsSysTime: 14s489ms - ScannerThreadsUserTime: 1m12s - ScannerThreadsVoluntaryContextSwitches: 61.11K (61111) - TotalRawHdfsReadTime(*): 25s375ms - TotalReadThroughput: 194.43 MB/sec Fragment F01: Instance d64b2ee4abd82c78:814dfd697b0710c0 (host=datanode-03.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 29s312ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.30 MB, 2.30 MB, 2.30 MB, 2.30 MB, 2.30 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 2.30 MB (2415640) - PerHostPeakMemUsage: 1.86 GB (1994939248) - PrepareTime: 181.920ms - RowsProduced: 280 (280) - TotalCpuTime: 263.564ms - TotalNetworkReceiveTime: 29s123ms - TotalNetworkSendTime: 5.448ms - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 48 (48) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 48.00 GB (51539607552) - PeakMemoryUsage: 784.00 KB (802816) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 259.821ms, non-child: 259.821ms, % non-child: 100.00%) - CodegenTime: 809.772us - CompileTime: 15.655ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 42 (42) - NumInstructions: 852 (852) - OptimizationTime: 63.295ms - PrepareTime: 180.366ms DataStreamSender (dst_id=9):(Total: 683.040us, non-child: 683.040us, % non-child: 100.00%) - BytesSent: 5.79 KB (5930) - NetworkThroughput(*): 13.21 MB/sec - OverallThroughput: 8.28 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - RowsReturned: 280 (280) - SerializeBatchTime: 242.612us - TransmitDataRPCTime: 428.164us - UncompressedRowBatchSize: 11.00 KB (11264) AGGREGATION_NODE (id=8):(Total: 29s306ms, non-child: 183.008ms, % non-child: 0.62%) ExecOption: Codegen Enabled - BuildTime: 91.392us - GetNewBlockTime: 64.284us - GetResultsTime: 85.376us - HTResizeTime: 2.700us - HashBuckets: 16.38K (16384) - LargestPartitionPercent: 8 (8) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.29 MB (2401560) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 280 (280) - RowsReturnedRate: 9.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 452.000ns EXCHANGE_NODE (id=7):(Total: 29s123ms, non-child: 40.000us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.11 KB, 2.11 KB, 4.04 KB, 4.04 KB, 4.04 KB - BytesReceived: 5.96 KB (6101) - ConvertRowBatchTime: 10.500us - DeserializeRowBatchTimer: 89.232us - FirstBatchArrivalWaitTime: 26s645ms - PeakMemoryUsage: 0 - RowsReturned: 342 (342) - RowsReturnedRate: 11.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance d64b2ee4abd82c78:814dfd697b0710be (host=datanode-01.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 29s309ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.29 MB, 2.29 MB, 2.30 MB, 2.30 MB, 2.30 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 2.30 MB (2415416) - PerHostPeakMemUsage: 1.86 GB (1994664344) - PrepareTime: 173.677ms - RowsProduced: 277 (277) - TotalCpuTime: 255.639ms - TotalNetworkReceiveTime: 29s129ms - TotalNetworkSendTime: 5.182ms - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 48 (48) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 48.00 GB (51539607552) - PeakMemoryUsage: 784.00 KB (802816) - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 252.589ms, non-child: 252.589ms, % non-child: 100.00%) - CodegenTime: 822.428us - CompileTime: 16.456ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 42 (42) - NumInstructions: 852 (852) - OptimizationTime: 63.503ms - PrepareTime: 172.157ms DataStreamSender (dst_id=9):(Total: 572.156us, non-child: 572.156us, % non-child: 100.00%) - BytesSent: 5.69 KB (5827) - NetworkThroughput(*): 15.59 MB/sec - OverallThroughput: 9.71 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - RowsReturned: 277 (277) - SerializeBatchTime: 194.860us - TransmitDataRPCTime: 356.460us - UncompressedRowBatchSize: 10.88 KB (11144) AGGREGATION_NODE (id=8):(Total: 29s303ms, non-child: 174.357ms, % non-child: 0.60%) ExecOption: Codegen Enabled - BuildTime: 43.760us - GetNewBlockTime: 80.428us - GetResultsTime: 54.628us - HTResizeTime: 1.812us - HashBuckets: 16.38K (16384) - LargestPartitionPercent: 8 (8) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.29 MB (2401408) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 277 (277) - RowsReturnedRate: 9.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 484.000ns EXCHANGE_NODE (id=7):(Total: 29s129ms, non-child: 24.364us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.89 KB, 1.89 KB, 3.83 KB, 3.83 KB, 3.83 KB - BytesReceived: 5.65 KB (5782) - ConvertRowBatchTime: 7.052us - DeserializeRowBatchTimer: 234.132us - FirstBatchArrivalWaitTime: 26s651ms - PeakMemoryUsage: 0 - RowsReturned: 321 (321) - RowsReturnedRate: 11.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Instance d64b2ee4abd82c78:814dfd697b0710bf (host=datanode-02.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 29s306ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 0, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.28 MB, 2.29 MB, 2.29 MB, 2.30 MB, 2.30 MB, 2.30 MB ThreadUsage(500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 2.30 MB (2415864) - PerHostPeakMemUsage: 1.48 GB (1593422664) - PrepareTime: 179.725ms - RowsProduced: 276 (276) - TotalCpuTime: 265.224ms - TotalNetworkReceiveTime: 29s121ms - TotalNetworkSendTime: 2.782ms - TotalStorageWaitTime: 0.000ns CodeGen:(Total: 261.655ms, non-child: 261.655ms, % non-child: 100.00%) - CodegenTime: 833.732us - CompileTime: 15.817ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 42 (42) - NumInstructions: 852 (852) - OptimizationTime: 67.149ms - PrepareTime: 178.211ms DataStreamSender (dst_id=9):(Total: 681.760us, non-child: 681.760us, % non-child: 100.00%) - BytesSent: 5.53 KB (5659) - NetworkThroughput(*): 11.54 MB/sec - OverallThroughput: 7.92 MB/sec - PeakMemoryUsage: 4.00 KB (4096) - RowsReturned: 276 (276) - SerializeBatchTime: 201.672us - TransmitDataRPCTime: 467.564us - UncompressedRowBatchSize: 10.84 KB (11104) AGGREGATION_NODE (id=8):(Total: 29s302ms, non-child: 180.834ms, % non-child: 0.62%) ExecOption: Codegen Enabled - BuildTime: 88.400us - GetNewBlockTime: 71.172us - GetResultsTime: 96.912us - HTResizeTime: 2.860us - HashBuckets: 16.38K (16384) - LargestPartitionPercent: 7 (7) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 2.29 MB (2401744) - PinTime: 0.000ns - RowsRepartitioned: 0 (0) - RowsReturned: 276 (276) - RowsReturnedRate: 9.00 /sec - SpilledPartitions: 0 (0) - UnpinTime: 484.000ns EXCHANGE_NODE (id=7):(Total: 29s121ms, non-child: 49.712us, % non-child: 0.00%) BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.88 KB, 1.88 KB, 3.94 KB, 3.94 KB, 3.94 KB - BytesReceived: 5.77 KB (5911) - ConvertRowBatchTime: 15.500us - DeserializeRowBatchTimer: 194.988us - FirstBatchArrivalWaitTime: 26s644ms - PeakMemoryUsage: 0 - RowsReturned: 337 (337) - RowsReturnedRate: 11.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Fragment F00: Instance d64b2ee4abd82c78:814dfd697b0710c1 (host=datanode-01.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 29s296ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:30/2.78 GB 1:34/3.47 GB MemoryUsage(500.000ms): 716.84 MB, 1.73 GB, 1.84 GB, 1.82 GB, 1.78 GB, 1.79 GB, 1.73 GB, 1.69 GB, 1.72 GB, 1.66 GB, 1.65 GB, 1.66 GB, 1.64 GB, 1.65 GB, 1.65 GB, 1.64 GB, 1.60 GB, 1.57 GB, 1.56 GB, 1.51 GB, 1.56 GB, 1.56 GB, 1.59 GB, 1.61 GB, 1.63 GB, 1.57 GB, 1.50 GB, 1.52 GB, 1.43 GB, 1.44 GB, 1.47 GB, 1.46 GB, 1.43 GB, 1.44 GB, 1.46 GB, 1.48 GB, 1.55 GB, 1.52 GB, 1.52 GB, 1.48 GB, 1.49 GB, 1.46 GB, 1.54 GB, 1.58 GB, 1.52 GB, 1.52 GB, 1.45 GB, 1.41 GB, 1.34 GB, 1.29 GB, 1.30 GB, 1.14 GB, 1.02 GB, 932.28 MB, 901.50 MB, 833.10 MB, 670.86 MB, 504.19 MB ThreadUsage(500.000ms): 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 21, 21, 19, 19, 19, 17, 14, 12, 12, 9, 6, 3 - AverageThreadTokens: 21.38 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.86 GB (1992271128) - PerHostPeakMemUsage: 1.86 GB (1994664344) - PrepareTime: 226.123ms - RowsProduced: 320 (320) - TotalCpuTime: 10m12s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.324ms - TotalStorageWaitTime: 5s872ms CodeGen:(Total: 273.549ms, non-child: 273.549ms, % non-child: 100.00%) - CodegenTime: 79.502ms - CompileTime: 20.109ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 52 (52) - NumInstructions: 1.11K (1106) - OptimizationTime: 68.300ms - PrepareTime: 184.712ms DataStreamSender (dst_id=7):(Total: 225.352us, non-child: 225.352us, % non-child: 100.00%) - BytesSent: 5.57 KB (5701) - NetworkThroughput(*): 41.92 MB/sec - OverallThroughput: 24.13 MB/sec - PeakMemoryUsage: 48.00 KB (49152) - RowsReturned: 320 (320) - SerializeBatchTime: 85.620us - TransmitDataRPCTime: 129.692us - UncompressedRowBatchSize: 12.51 KB (12812) AGGREGATION_NODE (id=6):(Total: 29s293ms, non-child: 9s515ms, % non-child: 32.48%) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 37.784us - GetResultsTime: 59.132us - HTResizeTime: 329.760us - HashBuckets: 32.77K (32768) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 40.05 MB (41997456) - PinTime: 0.000ns - ReductionFactorEstimate: 0.00 - ReductionFactorThresholdToExpand: 0.00 - RowsPassedThrough: 0 (0) - RowsReturned: 320 (320) - RowsReturnedRate: 10.00 /sec - StreamingTime: 9s309ms - UnpinTime: 0.000ns SUBPLAN_NODE (id=1):(Total: 19s777ms, non-child: 1s796ms, % non-child: 9.08%) - PeakMemoryUsage: 27.62 MB (28958304) - RowsReturned: 5.48M (5475448) - RowsReturnedRate: 276.85 K/sec NESTED_LOOP_JOIN_NODE (id=5):(Total: 17s900ms, non-child: 8s978ms, % non-child: 50.16%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 1.02 MB (1064960) - ProbeRows: 6.90M (6904194) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 AGGREGATION_NODE (id=4):(Total: 8s921ms, non-child: 7s785ms, % non-child: 87.26%) ExecOption: Codegen Disabled: Could not codegen PartitionedAggregationNode::UpdateTuple because intermediate type STRING is not yet supported for aggregate function "max()" - BuildTime: 2s781ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 2.02 MB (2113536) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) UNNEST_NODE (id=3):(Total: 1s136ms, non-child: 1s136ms, % non-child: 100.00%) - AvgCollectionSize: 3.00 - MaxCollectionSize: 3 (3) - MinCollectionSize: 3 (3) - NumCollections: 6.90M (6904194) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 80.589ms, non-child: 80.589ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 64 Hdfs split stats (:<# splits>/): 0:30/2.78 GB 1:34/3.47 GB Hdfs Read Thread Concurrency Bucket: 0:75.86% 1:15.52% 2:3.448% 3:3.448% 4:1.724% 5:0% 6:0% File Formats: PARQUET/NONE:192 BytesRead(500.000ms): 487.73 MB, 803.84 MB, 959.88 MB, 1.03 GB, 1.05 GB, 1.17 GB, 1.24 GB, 1.29 GB, 1.41 GB, 1.45 GB, 1.53 GB, 1.70 GB, 1.77 GB, 1.88 GB, 1.99 GB, 2.01 GB, 2.09 GB, 2.17 GB, 2.23 GB, 2.39 GB, 2.55 GB, 2.66 GB, 2.77 GB, 2.85 GB, 2.92 GB, 3.00 GB, 3.03 GB, 3.18 GB, 3.20 GB, 3.33 GB, 3.44 GB, 3.49 GB, 3.58 GB, 3.66 GB, 3.79 GB, 3.90 GB, 4.01 GB, 4.08 GB, 4.12 GB, 4.19 GB, 4.27 GB, 4.35 GB, 4.46 GB, 4.62 GB, 4.66 GB, 4.75 GB, 4.80 GB, 4.86 GB, 4.94 GB, 4.96 GB, 5.02 GB, 5.06 GB, 5.10 GB, 5.15 GB, 5.19 GB, 5.23 GB, 5.25 GB, 5.27 GB - AverageHdfsReadThreadConcurrency: 0.40 - AverageScannerThreadConcurrency: 20.38 - BytesRead: 5.28 GB (5673316444) - BytesReadDataNodeCache: 0 - BytesReadLocal: 5.28 GB (5673316444) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5.28 GB (5673316444) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 3 (3) - NumRowGroups: 64 (64) - NumScannerThreadsStarted: 23 (23) - PeakMemoryUsage: 1.83 GB (1962487960) - PerReadThreadRawHdfsThroughput: 559.25 MB/sec - RemoteScanRanges: 0 (0) - RowsRead: 575.76M (575763234) - RowsReturned: 6.90M (6904194) - RowsReturnedRate: 85.67 M/sec - ScanRangesComplete: 64 (64) - ScannerThreadsInvoluntaryContextSwitches: 675 (675) - ScannerThreadsTotalWallClockTime: 9m48s - MaterializeTupleTime(*): 9m38s - ScannerThreadsSysTime: 15s322ms - ScannerThreadsUserTime: 1m14s - ScannerThreadsVoluntaryContextSwitches: 60.27K (60271) - TotalRawHdfsReadTime(*): 9s674ms - TotalReadThroughput: 186.19 MB/sec Instance d64b2ee4abd82c78:814dfd697b0710c3 (host=datanode-03.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 27s733ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 1:26/2.65 GB 0:38/3.56 GB MemoryUsage(500.000ms): 1.31 MB, 1.56 GB, 1.70 GB, 1.71 GB, 1.69 GB, 1.67 GB, 1.67 GB, 1.70 GB, 1.70 GB, 1.69 GB, 1.66 GB, 1.70 GB, 1.71 GB, 1.72 GB, 1.76 GB, 1.64 GB, 1.73 GB, 1.74 GB, 1.71 GB, 1.78 GB, 1.72 GB, 1.67 GB, 1.68 GB, 1.63 GB, 1.66 GB, 1.64 GB, 1.59 GB, 1.61 GB, 1.58 GB, 1.66 GB, 1.62 GB, 1.71 GB, 1.72 GB, 1.66 GB, 1.70 GB, 1.70 GB, 1.68 GB, 1.79 GB, 1.77 GB, 1.79 GB, 1.74 GB, 1.77 GB, 1.69 GB, 1.70 GB, 1.62 GB, 1.54 GB, 1.43 GB, 1.41 GB, 1.35 GB, 1.26 GB, 1.14 GB, 1.04 GB, 934.77 MB, 827.87 MB, 618.95 MB, 104.04 MB ThreadUsage(500.000ms): 1, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 22, 22, 21, 19, 18, 17, 16, 15, 13, 12, 10, 8, 5, 1 - AverageThreadTokens: 21.14 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.86 GB (1992546032) - PerHostPeakMemUsage: 1.86 GB (1994939248) - PrepareTime: 207.568ms - RowsProduced: 348 (348) - TotalCpuTime: 9m33s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.602ms - TotalStorageWaitTime: 23s678ms CodeGen:(Total: 264.282ms, non-child: 264.282ms, % non-child: 100.00%) - CodegenTime: 51.716ms - CompileTime: 16.637ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 52 (52) - NumInstructions: 1.11K (1106) - OptimizationTime: 67.058ms - PrepareTime: 180.090ms DataStreamSender (dst_id=7):(Total: 251.324us, non-child: 251.324us, % non-child: 100.00%) - BytesSent: 5.94 KB (6079) - NetworkThroughput(*): 41.25 MB/sec - OverallThroughput: 23.07 MB/sec - PeakMemoryUsage: 48.00 KB (49152) - RowsReturned: 348 (348) - SerializeBatchTime: 98.476us - TransmitDataRPCTime: 140.536us - UncompressedRowBatchSize: 13.61 KB (13932) AGGREGATION_NODE (id=6):(Total: 27s730ms, non-child: 8s977ms, % non-child: 32.37%) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 33.796us - GetResultsTime: 58.308us - HTResizeTime: 286.784us - HashBuckets: 32.77K (32768) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 40.21 MB (42165424) - PinTime: 0.000ns - ReductionFactorEstimate: 0.00 - ReductionFactorThresholdToExpand: 0.00 - RowsPassedThrough: 0 (0) - RowsReturned: 348 (348) - RowsReturnedRate: 12.00 /sec - StreamingTime: 8s775ms - UnpinTime: 0.000ns SUBPLAN_NODE (id=1):(Total: 18s753ms, non-child: 1s739ms, % non-child: 9.27%) - PeakMemoryUsage: 23.34 MB (24473216) - RowsReturned: 4.78M (4775782) - RowsReturnedRate: 254.66 K/sec NESTED_LOOP_JOIN_NODE (id=5):(Total: 16s916ms, non-child: 8s580ms, % non-child: 50.72%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 1.02 MB (1064960) - ProbeRows: 6.42M (6424876) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 AGGREGATION_NODE (id=4):(Total: 8s335ms, non-child: 7s292ms, % non-child: 87.49%) ExecOption: Codegen Disabled: Could not codegen PartitionedAggregationNode::UpdateTuple because intermediate type STRING is not yet supported for aggregate function "max()" - BuildTime: 2s605ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 2.02 MB (2113536) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) UNNEST_NODE (id=3):(Total: 1s042ms, non-child: 1s042ms, % non-child: 100.00%) - AvgCollectionSize: 3.00 - MaxCollectionSize: 3 (3) - MinCollectionSize: 3 (3) - NumCollections: 6.42M (6424876) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 97.391ms, non-child: 97.391ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 64 Hdfs split stats (:<# splits>/): 1:26/2.65 GB 0:38/3.56 GB Hdfs Read Thread Concurrency Bucket: 0:52.73% 1:9.091% 2:7.273% 3:27.27% 4:1.818% 5:1.818% 6:0% 7:0% File Formats: PARQUET/NONE:192 BytesRead(500.000ms): 0, 573.02 MB, 704.17 MB, 808.89 MB, 931.80 MB, 1.00 GB, 1.15 GB, 1.25 GB, 1.34 GB, 1.45 GB, 1.50 GB, 1.66 GB, 1.76 GB, 1.85 GB, 1.94 GB, 1.98 GB, 2.09 GB, 2.28 GB, 2.42 GB, 2.58 GB, 2.61 GB, 2.67 GB, 2.78 GB, 2.85 GB, 2.95 GB, 3.08 GB, 3.10 GB, 3.20 GB, 3.28 GB, 3.43 GB, 3.47 GB, 3.71 GB, 3.81 GB, 3.86 GB, 3.96 GB, 4.05 GB, 4.12 GB, 4.26 GB, 4.31 GB, 4.39 GB, 4.46 GB, 4.57 GB, 4.62 GB, 4.69 GB, 4.76 GB, 4.81 GB, 4.85 GB, 4.95 GB, 5.01 GB, 5.05 GB, 5.11 GB, 5.15 GB, 5.20 GB, 5.23 GB, 5.24 GB, 5.25 GB - AverageHdfsReadThreadConcurrency: 1.22 - AverageScannerThreadConcurrency: 20.51 - BytesRead: 5.25 GB (5642443431) - BytesReadDataNodeCache: 0 - BytesReadLocal: 5.25 GB (5642443431) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5.25 GB (5642443431) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 3 (3) - NumRowGroups: 64 (64) - NumScannerThreadsStarted: 23 (23) - PeakMemoryUsage: 1.83 GB (1965361376) - PerReadThreadRawHdfsThroughput: 159.00 MB/sec - RemoteScanRanges: 0 (0) - RowsRead: 572.59M (572589332) - RowsReturned: 6.42M (6424876) - RowsReturnedRate: 65.97 M/sec - ScanRangesComplete: 64 (64) - ScannerThreadsInvoluntaryContextSwitches: 443 (443) - ScannerThreadsTotalWallClockTime: 9m29s - MaterializeTupleTime(*): 9m - ScannerThreadsSysTime: 14s406ms - ScannerThreadsUserTime: 1m12s - ScannerThreadsVoluntaryContextSwitches: 59.65K (59649) - TotalRawHdfsReadTime(*): 33s842ms - TotalReadThroughput: 192.18 MB/sec Instance d64b2ee4abd82c78:814dfd697b0710c2 (host=datanode-02.zamboni.test.hadoop.bol.nordeus.com:22000):(Total: 26s822ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:37/3.52 GB 1:26/2.77 GB MemoryUsage(500.000ms): 0, 1.22 GB, 1.44 GB, 1.41 GB, 1.37 GB, 1.37 GB, 1.35 GB, 1.35 GB, 1.32 GB, 1.34 GB, 1.29 GB, 1.20 GB, 1.15 GB, 1.08 GB, 995.35 MB, 1.07 GB, 1.12 GB, 1.14 GB, 1.18 GB, 1.18 GB, 1.16 GB, 1.15 GB, 1.00 GB, 956.39 MB, 1.02 GB, 1.08 GB, 1.10 GB, 1.12 GB, 1.18 GB, 1.12 GB, 1.14 GB, 1.17 GB, 1.19 GB, 1.10 GB, 1.18 GB, 1.25 GB, 1.20 GB, 1.13 GB, 1.11 GB, 1.10 GB, 1.08 GB, 1.10 GB, 1.21 GB, 1.15 GB, 1.13 GB, 1.18 GB, 1.10 GB, 1.06 GB, 1.05 GB, 1.01 GB, 965.58 MB, 849.86 MB, 750.48 MB, 577.05 MB ThreadUsage(500.000ms): 1, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 13, 12, 11, 11, 11, 8, 7, 4 - AverageThreadTokens: 14.46 - BloomFilterBytes: 0 - PeakMemoryUsage: 1.48 GB (1591021256) - PerHostPeakMemUsage: 1.48 GB (1593422664) - PrepareTime: 212.402ms - RowsProduced: 332 (332) - TotalCpuTime: 6m3s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 1.094ms - TotalStorageWaitTime: 44s973ms CodeGen:(Total: 267.633ms, non-child: 267.633ms, % non-child: 100.00%) - CodegenTime: 57.117ms - CompileTime: 16.789ms - LoadTime: 0.000ns - ModuleBitcodeSize: 2.15 MB (2254656) - NumFunctions: 52 (52) - NumInstructions: 1.11K (1106) - OptimizationTime: 68.217ms - PrepareTime: 182.213ms DataStreamSender (dst_id=7):(Total: 237.616us, non-child: 237.616us, % non-child: 100.00%) - BytesSent: 5.87 KB (6014) - NetworkThroughput(*): 55.44 MB/sec - OverallThroughput: 24.14 MB/sec - PeakMemoryUsage: 48.00 KB (49152) - RowsReturned: 332 (332) - SerializeBatchTime: 112.280us - TransmitDataRPCTime: 103.452us - UncompressedRowBatchSize: 12.98 KB (13292) AGGREGATION_NODE (id=6):(Total: 26s819ms, non-child: 8s479ms, % non-child: 31.62%) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 38.204us - GetResultsTime: 58.312us - HTResizeTime: 276.460us - HashBuckets: 32.77K (32768) - LargestPartitionPercent: 0 (0) - PartitionsCreated: 16 (16) - PeakMemoryUsage: 41.58 MB (43595256) - PinTime: 0.000ns - ReductionFactorEstimate: 0.00 - ReductionFactorThresholdToExpand: 0.00 - RowsPassedThrough: 0 (0) - RowsReturned: 332 (332) - RowsReturnedRate: 12.00 /sec - StreamingTime: 8s276ms - UnpinTime: 0.000ns SUBPLAN_NODE (id=1):(Total: 18s340ms, non-child: 1s662ms, % non-child: 9.06%) - PeakMemoryUsage: 26.92 MB (28225216) - RowsReturned: 5.32M (5320108) - RowsReturnedRate: 290.08 K/sec NESTED_LOOP_JOIN_NODE (id=5):(Total: 16s594ms, non-child: 7s891ms, % non-child: 47.56%) - BuildRows: 0 (0) - BuildTime: 0.000ns - PeakMemoryUsage: 1.02 MB (1064960) - ProbeRows: 6.73M (6733787) - ProbeTime: 0.000ns - RowsReturned: 0 (0) - RowsReturnedRate: 0 SINGULAR_ROW_SRC_NODE (id=2): - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 AGGREGATION_NODE (id=4):(Total: 8s702ms, non-child: 7s606ms, % non-child: 87.40%) ExecOption: Codegen Disabled: Could not codegen PartitionedAggregationNode::UpdateTuple because intermediate type STRING is not yet supported for aggregate function "max()" - BuildTime: 2s678ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 2.02 MB (2113536) - RowsRepartitioned: 0 (0) - RowsReturned: 0 (0) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) UNNEST_NODE (id=3):(Total: 1s096ms, non-child: 1s096ms, % non-child: 100.00%) - AvgCollectionSize: 3.00 - MaxCollectionSize: 3 (3) - MinCollectionSize: 3 (3) - NumCollections: 6.73M (6733787) - PeakMemoryUsage: 0 - RowsReturned: 0 (0) - RowsReturnedRate: 0 HDFS_SCAN_NODE (id=0):(Total: 83.112ms, non-child: 83.112ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 63 Hdfs split stats (:<# splits>/): 0:37/3.52 GB 1:26/2.77 GB Hdfs Read Thread Concurrency Bucket: 0:41.51% 1:15.09% 2:28.3% 3:13.21% 4:1.887% 5:0% 6:0% File Formats: PARQUET/NONE:189 BytesRead(500.000ms): 353.21 MB, 652.21 MB, 716.21 MB, 789.06 MB, 939.86 MB, 1003.86 MB, 1.09 GB, 1.16 GB, 1.24 GB, 1.36 GB, 1.39 GB, 1.48 GB, 1.61 GB, 1.67 GB, 1.82 GB, 1.92 GB, 2.00 GB, 2.11 GB, 2.20 GB, 2.30 GB, 2.38 GB, 2.45 GB, 2.65 GB, 2.73 GB, 2.86 GB, 2.97 GB, 3.10 GB, 3.26 GB, 3.31 GB, 3.40 GB, 3.52 GB, 3.63 GB, 3.69 GB, 3.83 GB, 3.95 GB, 4.00 GB, 4.06 GB, 4.11 GB, 4.26 GB, 4.35 GB, 4.42 GB, 4.59 GB, 4.63 GB, 4.73 GB, 4.82 GB, 4.91 GB, 4.97 GB, 5.05 GB, 5.13 GB, 5.17 GB, 5.23 GB, 5.29 GB, 5.30 GB - AverageHdfsReadThreadConcurrency: 1.19 - AverageScannerThreadConcurrency: 13.72 - BytesRead: 5.31 GB (5697577107) - BytesReadDataNodeCache: 0 - BytesReadLocal: 5.31 GB (5697577107) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 5.31 GB (5697577107) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 3 (3) - NumDisksAccessed: 3 (3) - NumRowGroups: 62 (62) - NumScannerThreadsStarted: 15 (15) - PeakMemoryUsage: 1.46 GB (1564301896) - PerReadThreadRawHdfsThroughput: 166.63 MB/sec - RemoteScanRanges: 0 (0) - RowsRead: 577.67M (577667760) - RowsReturned: 6.73M (6733787) - RowsReturnedRate: 81.02 M/sec - ScanRangesComplete: 63 (63) - ScannerThreadsInvoluntaryContextSwitches: 312 (312) - ScannerThreadsTotalWallClockTime: 6m3s - MaterializeTupleTime(*): 5m15s - ScannerThreadsSysTime: 13s739ms - ScannerThreadsUserTime: 1m10s - ScannerThreadsVoluntaryContextSwitches: 63.41K (63413) - TotalRawHdfsReadTime(*): 32s609ms - TotalReadThroughput: 204.91 MB/sec