Query (id=204ae54f7a5c9622:ce67ab39615d87bb): Summary: Session ID: b744ad5dad86e615:f03d70ca411347ad Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2015-03-06 15:38:00.283456000 End Time: 2015-03-06 15:38:02.929707000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.1.2-cdh5 RELEASE (build 92438b7fb62a875041ffd2ab324f408ecf8129e5) User: d.lamanov@plarium.local Connected User: d.lamanov@plarium.local Delegated User: Network Address: 192.168.103.98:49537 Default Db: dwh Sql Statement: SELECT DISTINCT YEAR(`DimUser`.`regtime`) as `EXPR_0` FROM `dwh`.`DimUser` Coordinator: node01:22000 Plan: ---------------- Estimated Per-Host Requirements: Memory=256.00MB VCores=2 WARNING: The following tables are missing relevant table and/or column statistics. dwh.dimuser F02:PLAN FRAGMENT [UNPARTITIONED] 04:EXCHANGE [UNPARTITIONED] hosts=3 per-host-mem=unavailable tuple-ids=1 row-size=4B cardinality=unavailable F01:PLAN FRAGMENT [HASH(year(DimUser.regtime))] DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=04, UNPARTITIONED] 03:AGGREGATE [FINALIZE] | group by: year(DimUser.regtime) | hosts=3 per-host-mem=128.00MB | tuple-ids=1 row-size=4B cardinality=unavailable | 02:EXCHANGE [HASH(year(DimUser.regtime))] hosts=3 per-host-mem=0B tuple-ids=1 row-size=4B cardinality=unavailable F00:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F01, EXCHANGE=02, HASH(year(DimUser.regtime))] 01:AGGREGATE | group by: year(DimUser.regtime) | hosts=3 per-host-mem=128.00MB | tuple-ids=1 row-size=4B cardinality=unavailable | 00:SCAN HDFS [dwh.dimuser, RANDOM] partitions=1/1 files=41 size=9.42GB table stats: unavailable column stats: unavailable hosts=3 per-host-mem=88.00MB tuple-ids=0 row-size=16B cardinality=unavailable ---------------- Estimated Per-Host Mem: 268435456 Estimated Per-Host VCores: 2 Tables Missing Stats: dwh.dimuser Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------ 04:EXCHANGE 1 31.349us 31.349us 6 -1 0 -1.00 B UNPARTITIONED 03:AGGREGATE 3 171.141ms 173.549ms 6 -1 6.27 MB 128.00 MB FINALIZE 02:EXCHANGE 3 95.707us 164.549us 18 -1 0 0 HASH(year(DimUser.regtime)) 01:AGGREGATE 3 1s886ms 1s912ms 18 -1 6.81 MB 128.00 MB 00:SCAN HDFS 3 158.71ms 164.661ms 140.39M -1 351.82 MB 88.00 MB dwh.dimuser Query Timeline: 2s649ms - Start execution: 69.302us (69.302us) - Planning finished: 14.217ms (14.147ms) - Ready to start remote fragments: 14.605ms (387.861us) - Remote fragments started: 360.145ms (345.540ms) - Rows available: 2s641ms (2s281ms) - First row fetched: 2s643ms (2.266ms) - Unregister query: 2s646ms (2.669ms) ImpalaServer: - ClientFetchWaitTimer: 4.66ms - RowMaterializationTimer: 9.456us Execution Profile 204ae54f7a5c9622:ce67ab39615d87bb:(Total: 2s627ms, non-child: 0ns, % non-child: 0.00%) Fragment start latencies: count: 6, last: 0.170542, min: 0.168309, max: 0.174175, mean: 0.170379, stddev: 0.00223581 Per Node Peak Memory Usage: node03:22000(364.49 MB) node02:22000(361.90 MB) node01:22000(312.00 MB) - FinalizationTimer: 0ns Coordinator Fragment F02:(Total: 2s281ms, non-child: 140.582us, % non-child: 0.01%) MemoryUsage(500.0ms): 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB, 8.00 KB - AverageThreadTokens: 0.00 - PeakMemoryUsage: 36.00 KB (36864) - PerHostPeakMemUsage: 0 - PrepareTime: 24.903us - RowsProduced: 6 - TotalCpuTime: 349.300ms - TotalNetworkReceiveTime: 2s280ms - TotalNetworkSendTime: 0ns - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 - BlocksCreated: 192 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 43.20 GB (46385647616) - PeakMemoryUsage: 512.00 KB (524288) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns EXCHANGE_NODE (id=4):(Total: 2s280ms, non-child: 31.349us, % non-child: 0.00%) BytesReceived(500.0ms): 0, 0, 0, 0, 0 - BytesReceived: 96.00 B (96) - ConvertRowBatchTime: 5.756us - DeserializeRowBatchTimer: 15.91us - FirstBatchArrivalWaitTime: 2s280ms - PeakMemoryUsage: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Averaged Fragment F01:(Total: 2s402ms, non-child: 0ns, % non-child: 0.00%) split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:2s453ms max:2s459ms mean: 2s456ms stddev:2.193ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 6.30 MB (6602752) - PerHostPeakMemUsage: 346.13 MB (362947224) - PrepareTime: 170.788ms - RowsProduced: 2 - TotalCpuTime: 395.549ms - TotalNetworkReceiveTime: 2s231ms - TotalNetworkSendTime: 445.270us - TotalStorageWaitTime: 0ns CodeGen:(Total: 392.244ms, non-child: 392.244ms, % non-child: 100.00%) - CodegenTime: 1.787ms - CompileTime: 27.664ms - LoadTime: 30.333us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 195.894ms - PrepareTime: 167.407ms DataStreamSender (dst_id=4):(Total: 374.802us, non-child: 374.802us, % non-child: 100.00%) - BytesSent: 36.00 B (36) - NetworkThroughput(*): 103.63 KB/sec - OverallThroughput: 115.79 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 9.370us - ThriftTransmitTime(*): 371.230us - UncompressedRowBatchSize: 36.00 B (36) AGGREGATION_NODE (id=3):(Total: 2s402ms, non-child: 171.141ms, % non-child: 7.12%) - BuildTime: 9.61us - GetNewBlockTime: 63.695us - GetResultsTime: 2.973us - HashBuckets: 2.05K (2048) - LargestPartitionPercent: 20 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.27 MB (6572714) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 2 - RowsReturnedRate: 0 - SpilledPartitions: 0 - UnpinTime: 1.893us EXCHANGE_NODE (id=2):(Total: 2s231ms, non-child: 2s231ms, % non-child: 100.00%) - BytesReceived: 84.00 B (84) - ConvertRowBatchTime: 4.311us - DeserializeRowBatchTimer: 12.109us - FirstBatchArrivalWaitTime: 2s198ms - PeakMemoryUsage: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns BlockMgr: - BlockWritesOutstanding: 0 - BlocksCreated: 192 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 43.20 GB (46385647616) - PeakMemoryUsage: 512.00 KB (524288) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns Averaged Fragment F00:(Total: 2s045ms, non-child: 0ns, % non-child: 0.00%) split sizes: min: 3.11 GB, max: 3.16 GB, avg: 3.14 GB, stddev: 18.98 MB completion times: min:2s252ms max:2s284ms mean: 2s267ms stddev:13.290ms execution rates: min:1.38 GB/sec max:1.40 GB/sec mean:1.38 GB/sec stddev:8.92 MB/sec num instances: 3 - AverageThreadTokens: 10.27 - PeakMemoryUsage: 339.86 MB (356366317) - PerHostPeakMemUsage: 346.13 MB (362947224) - PrepareTime: 168.219ms - RowsProduced: 6 - TotalCpuTime: 23s398ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 583.159us - TotalStorageWaitTime: 786.765ms CodeGen:(Total: 554.255ms, non-child: 554.255ms, % non-child: 100.00%) - CodegenTime: 1.862ms - CompileTime: 98.120ms - LoadTime: 26.326us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 290.341ms - PrepareTime: 164.576ms DataStreamSender (dst_id=2):(Total: 25.135us, non-child: 25.135us, % non-child: 100.00%) - BytesSent: 84.00 B (84) - NetworkThroughput(*): 173.49 KB/sec - OverallThroughput: 3.19 MB/sec - PeakMemoryUsage: 144.00 KB (147456) - SerializeBatchTime: 15.887us - ThriftTransmitTime(*): 481.94us - UncompressedRowBatchSize: 84.00 B (84) AGGREGATION_NODE (id=1):(Total: 2s044ms, non-child: 1s886ms, % non-child: 92.27%) - BuildTime: 1s690ms - GetNewBlockTime: 67.375us - GetResultsTime: 6.79us - HashBuckets: 6.14K (6144) - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.81 MB (7144202) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 - UnpinTime: 1.910us HDFS_SCAN_NODE (id=0):(Total: 158.71ms, non-child: 158.71ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.17 - AverageScannerThreadConcurrency: 11.58 - BytesRead: 331.13 MB (347215497) - BytesReadDataNodeCache: 0 - BytesReadLocal: 331.13 MB (347215497) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 331.13 MB (347215497) - DecompressionTime: 731.401ms - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 13 - PeakMemoryUsage: 333.49 MB (349692565) - PerReadThreadRawHdfsThroughput: 2.15 GB/sec - RemoteScanRanges: 0 - RowsRead: 46.80M (46797944) - RowsReturned: 46.80M (46797944) - RowsReturnedRate: 296.45 M/sec - ScanRangesComplete: 13 - ScannerThreadsInvoluntaryContextSwitches: 566 - ScannerThreadsTotalWallClockTime: 21s750ms - MaterializeTupleTime(*): 19s919ms - ScannerThreadsSysTime: 113.390ms - ScannerThreadsUserTime: 2s813ms - ScannerThreadsVoluntaryContextSwitches: 46.53K (46532) - TotalRawHdfsReadTime(*): 171.277ms - TotalReadThroughput: 132.45 MB/sec Fragment F01: Instance 204ae54f7a5c9622:ce67ab39615d87bd (host=node02:22000):(Total: 2s405ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 6.30 MB (6602752) - PerHostPeakMemUsage: 361.90 MB (379480808) - PrepareTime: 173.191ms - RowsProduced: 3 - TotalCpuTime: 395.498ms - TotalNetworkReceiveTime: 2s230ms - TotalNetworkSendTime: 762.866us - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 - BlocksCreated: 192 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 43.20 GB (46385647616) - PeakMemoryUsage: 512.00 KB (524288) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 392.163ms, non-child: 392.163ms, % non-child: 100.00%) - CodegenTime: 1.767ms - CompileTime: 27.81ms - LoadTime: 33.333us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 193.971ms - PrepareTime: 169.783ms DataStreamSender (dst_id=4):(Total: 694.733us, non-child: 694.733us, % non-child: 100.00%) - BytesSent: 52.00 B (52) - NetworkThroughput(*): 78.39 KB/sec - OverallThroughput: 73.09 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 14.443us - ThriftTransmitTime(*): 647.825us - UncompressedRowBatchSize: 52.00 B (52) AGGREGATION_NODE (id=3):(Total: 2s404ms, non-child: 173.549ms, % non-child: 7.22%) ExecOption: Codegen Enabled - BuildTime: 11.146us - GetNewBlockTime: 59.993us - GetResultsTime: 4.137us - HashBuckets: 3.07K (3072) - LargestPartitionPercent: 11 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.27 MB (6574080) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 3 - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 - UnpinTime: 1.875us EXCHANGE_NODE (id=2):(Total: 2s230ms, non-child: 164.549us, % non-child: 0.01%) BytesReceived(500.0ms): 0, 0, 0, 0, 0 - BytesReceived: 120.00 B (120) - ConvertRowBatchTime: 5.701us - DeserializeRowBatchTimer: 15.733us - FirstBatchArrivalWaitTime: 2s198ms - PeakMemoryUsage: 0 - RowsReturned: 9 - RowsReturnedRate: 4.00 /sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 204ae54f7a5c9622:ce67ab39615d87be (host=node03:22000):(Total: 2s403ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 6.30 MB (6602752) - PerHostPeakMemUsage: 364.49 MB (382200552) - PrepareTime: 171.290ms - RowsProduced: 2 - TotalCpuTime: 395.119ms - TotalNetworkReceiveTime: 2s231ms - TotalNetworkSendTime: 368.434us - TotalStorageWaitTime: 0ns BlockMgr: - BlockWritesOutstanding: 0 - BlocksCreated: 192 - BlocksRecycled: 0 - BufferedPins: 0 - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 43.20 GB (46385647616) - PeakMemoryUsage: 512.00 KB (524288) - TotalBufferWaitTime: 0ns - TotalEncryptionTime: 0ns - TotalIntegrityCheckTime: 0ns - TotalReadBlockTime: 0ns CodeGen:(Total: 391.669ms, non-child: 391.669ms, % non-child: 100.00%) - CodegenTime: 1.783ms - CompileTime: 27.99ms - LoadTime: 32.272us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 195.377ms - PrepareTime: 167.933ms DataStreamSender (dst_id=4):(Total: 307.588us, non-child: 307.588us, % non-child: 100.00%) - BytesSent: 36.00 B (36) - NetworkThroughput(*): 120.84 KB/sec - OverallThroughput: 114.30 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 8.897us - ThriftTransmitTime(*): 290.941us - UncompressedRowBatchSize: 36.00 B (36) AGGREGATION_NODE (id=3):(Total: 2s403ms, non-child: 171.701ms, % non-child: 7.14%) ExecOption: Codegen Enabled - BuildTime: 8.740us - GetNewBlockTime: 62.206us - GetResultsTime: 3.34us - HashBuckets: 2.05K (2048) - LargestPartitionPercent: 16 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.27 MB (6574080) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 2 - RowsReturnedRate: 0 - SpilledPartitions: 0 - UnpinTime: 1.894us EXCHANGE_NODE (id=2):(Total: 2s231ms, non-child: 40.287us, % non-child: 0.00%) BytesReceived(500.0ms): 0, 0, 0, 0, 0 - BytesReceived: 84.00 B (84) - ConvertRowBatchTime: 3.452us - DeserializeRowBatchTimer: 10.183us - FirstBatchArrivalWaitTime: 2s198ms - PeakMemoryUsage: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Instance 204ae54f7a5c9622:ce67ab39615d87bf (host=node01:22000):(Total: 2s399ms, non-child: 0ns, % non-child: 0.00%) MemoryUsage(500.0ms): 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB, 6.27 MB ThreadUsage(500.0ms): 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - PeakMemoryUsage: 6.30 MB (6602752) - PerHostPeakMemUsage: 312.00 MB (327160312) - PrepareTime: 167.884ms - RowsProduced: 1 - TotalCpuTime: 396.30ms - TotalNetworkReceiveTime: 2s230ms - TotalNetworkSendTime: 204.512us - TotalStorageWaitTime: 0ns CodeGen:(Total: 392.899ms, non-child: 392.899ms, % non-child: 100.00%) - CodegenTime: 1.811ms - CompileTime: 28.812ms - LoadTime: 25.396us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 198.333ms - PrepareTime: 164.505ms DataStreamSender (dst_id=4):(Total: 122.86us, non-child: 122.86us, % non-child: 100.00%) - BytesSent: 20.00 B (20) - NetworkThroughput(*): 111.66 KB/sec - OverallThroughput: 159.98 KB/sec - PeakMemoryUsage: 16.00 KB (16384) - SerializeBatchTime: 4.771us - ThriftTransmitTime(*): 174.924us - UncompressedRowBatchSize: 20.00 B (20) AGGREGATION_NODE (id=3):(Total: 2s399ms, non-child: 168.171ms, % non-child: 7.01%) ExecOption: Codegen Enabled - BuildTime: 7.299us - GetNewBlockTime: 68.886us - GetResultsTime: 1.750us - HashBuckets: 1.02K (1024) - LargestPartitionPercent: 33 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.27 MB (6569984) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SpilledPartitions: 0 - UnpinTime: 1.910us EXCHANGE_NODE (id=2):(Total: 2s230ms, non-child: 82.285us, % non-child: 0.00%) BytesReceived(500.0ms): 0, 0, 0, 0, 0 - BytesReceived: 48.00 B (48) - ConvertRowBatchTime: 3.782us - DeserializeRowBatchTimer: 10.413us - FirstBatchArrivalWaitTime: 2s198ms - PeakMemoryUsage: 0 - RowsReturned: 3 - RowsReturnedRate: 1.00 /sec - SendersBlockedTimer: 0ns - SendersBlockedTotalTimer(*): 0ns Fragment F00: Instance 204ae54f7a5c9622:ce67ab39615d87c1 (host=node03:22000):(Total: 2s063ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:14/3.16 GB MemoryUsage(500.0ms): 6.30 MB, 349.40 MB, 261.91 MB, 198.29 MB, 116.88 MB ThreadUsage(500.0ms): 1, 14, 14, 14, 13 - AverageThreadTokens: 11.20 - PeakMemoryUsage: 358.22 MB (375622376) - PerHostPeakMemUsage: 364.49 MB (382200552) - PrepareTime: 167.558ms - RowsProduced: 6 - TotalCpuTime: 25s115ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 624.601us - TotalStorageWaitTime: 728.353ms CodeGen:(Total: 552.278ms, non-child: 552.278ms, % non-child: 100.00%) - CodegenTime: 1.826ms - CompileTime: 97.846ms - LoadTime: 22.445us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 289.260ms - PrepareTime: 163.975ms DataStreamSender (dst_id=2):(Total: 25.385us, non-child: 25.385us, % non-child: 100.00%) - BytesSent: 84.00 B (84) - NetworkThroughput(*): 178.44 KB/sec - OverallThroughput: 3.16 MB/sec - PeakMemoryUsage: 144.00 KB (147456) - SerializeBatchTime: 15.410us - ThriftTransmitTime(*): 459.707us - UncompressedRowBatchSize: 84.00 B (84) AGGREGATION_NODE (id=1):(Total: 2s062ms, non-child: 1s912ms, % non-child: 92.69%) ExecOption: Codegen Enabled - BuildTime: 1s715ms - GetNewBlockTime: 67.346us - GetResultsTime: 6.904us - HashBuckets: 6.14K (6144) - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.81 MB (7144200) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 - UnpinTime: 1.909us HDFS_SCAN_NODE (id=0):(Total: 150.708ms, non-child: 150.708ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:14/3.16 GB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% File Formats: PARQUET/SNAPPY:14 ExecOption: Codegen enabled: 0 out of 14 BytesRead(500.0ms): 0, 332.81 MB, 332.81 MB, 332.81 MB, 332.81 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 12.75 - BytesRead: 332.81 MB (348977630) - BytesReadDataNodeCache: 0 - BytesReadLocal: 332.81 MB (348977630) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 332.81 MB (348977630) - DecompressionTime: 734.933ms - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 14 - PeakMemoryUsage: 351.82 MB (368904928) - PerReadThreadRawHdfsThroughput: 1.22 GB/sec - RemoteScanRanges: 0 - RowsRead: 46.80M (46795783) - RowsReturned: 46.80M (46795783) - RowsReturnedRate: 310.51 M/sec - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 581 - ScannerThreadsTotalWallClockTime: 23s392ms - MaterializeTupleTime(*): 21s808ms - ScannerThreadsSysTime: 92.871ms - ScannerThreadsUserTime: 2s822ms - ScannerThreadsVoluntaryContextSwitches: 46.20K (46202) - TotalRawHdfsReadTime(*): 266.829ms - TotalReadThroughput: 133.12 MB/sec Instance 204ae54f7a5c9622:ce67ab39615d87c2 (host=node01:22000):(Total: 2s044ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:13/3.11 GB MemoryUsage(500.0ms): 6.30 MB, 189.79 MB, 234.18 MB, 146.29 MB, 69.06 MB ThreadUsage(500.0ms): 1, 12, 12, 12, 5 - AverageThreadTokens: 8.40 - PeakMemoryUsage: 305.72 MB (320573944) - PerHostPeakMemUsage: 312.00 MB (327160312) - PrepareTime: 169.676ms - RowsProduced: 6 - TotalCpuTime: 19s608ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 476.696us - TotalStorageWaitTime: 648.489ms CodeGen:(Total: 556.192ms, non-child: 556.192ms, % non-child: 100.00%) - CodegenTime: 1.915ms - CompileTime: 98.488ms - LoadTime: 35.956us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 290.520ms - PrepareTime: 165.959ms DataStreamSender (dst_id=2):(Total: 25.925us, non-child: 25.925us, % non-child: 100.00%) - BytesSent: 84.00 B (84) - NetworkThroughput(*): 197.88 KB/sec - OverallThroughput: 3.09 MB/sec - PeakMemoryUsage: 144.00 KB (147456) - SerializeBatchTime: 13.346us - ThriftTransmitTime(*): 414.558us - UncompressedRowBatchSize: 84.00 B (84) AGGREGATION_NODE (id=1):(Total: 2s043ms, non-child: 1s879ms, % non-child: 91.94%) ExecOption: Codegen Enabled - BuildTime: 1s681ms - GetNewBlockTime: 65.678us - GetResultsTime: 5.719us - HashBuckets: 6.14K (6144) - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.81 MB (7144200) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 - UnpinTime: 1.925us HDFS_SCAN_NODE (id=0):(Total: 164.661ms, non-child: 164.661ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:13/3.11 GB Hdfs Read Thread Concurrency Bucket: 0:75% 1:25% 2:0% 3:0% File Formats: PARQUET/SNAPPY:13 ExecOption: Codegen enabled: 0 out of 13 BytesRead(500.0ms): 0, 164.50 MB, 278.72 MB, 278.72 MB, 329.04 MB - AverageHdfsReadThreadConcurrency: 0.25 - AverageScannerThreadConcurrency: 9.25 - BytesRead: 329.04 MB (345025082) - BytesReadDataNodeCache: 0 - BytesReadLocal: 329.04 MB (345025082) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 329.04 MB (345025082) - DecompressionTime: 733.338ms - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 11 - PeakMemoryUsage: 299.38 MB (313922040) - PerReadThreadRawHdfsThroughput: 2.60 GB/sec - RemoteScanRanges: 0 - RowsRead: 46.71M (46710328) - RowsReturned: 46.71M (46710328) - RowsReturnedRate: 283.68 M/sec - ScanRangesComplete: 13 - ScannerThreadsInvoluntaryContextSwitches: 564 - ScannerThreadsTotalWallClockTime: 17s823ms - MaterializeTupleTime(*): 16s131ms - ScannerThreadsSysTime: 107.830ms - ScannerThreadsUserTime: 2s919ms - ScannerThreadsVoluntaryContextSwitches: 46.41K (46410) - TotalRawHdfsReadTime(*): 123.730ms - TotalReadThroughput: 131.62 MB/sec Instance 204ae54f7a5c9622:ce67ab39615d87c0 (host=node02:22000):(Total: 2s028ms, non-child: 0ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:14/3.14 GB MemoryUsage(500.0ms): 6.30 MB, 249.05 MB, 299.88 MB, 211.66 MB, 101.50 MB ThreadUsage(500.0ms): 1, 15, 15, 14, 11 - AverageThreadTokens: 11.20 - PeakMemoryUsage: 355.63 MB (372902632) - PerHostPeakMemUsage: 361.90 MB (379480808) - PrepareTime: 167.425ms - RowsProduced: 6 - TotalCpuTime: 25s470ms - TotalNetworkReceiveTime: 0ns - TotalNetworkSendTime: 648.181us - TotalStorageWaitTime: 983.452ms CodeGen:(Total: 554.293ms, non-child: 554.293ms, % non-child: 100.00%) - CodegenTime: 1.846ms - CompileTime: 98.25ms - LoadTime: 20.579us - ModuleFileSize: 1.98 MB (2073836) - OptimizationTime: 291.242ms - PrepareTime: 163.794ms DataStreamSender (dst_id=2):(Total: 24.97us, non-child: 24.97us, % non-child: 100.00%) - BytesSent: 84.00 B (84) - NetworkThroughput(*): 144.16 KB/sec - OverallThroughput: 3.32 MB/sec - PeakMemoryUsage: 144.00 KB (147456) - SerializeBatchTime: 18.907us - ThriftTransmitTime(*): 569.18us - UncompressedRowBatchSize: 84.00 B (84) AGGREGATION_NODE (id=1):(Total: 2s028ms, non-child: 1s869ms, % non-child: 92.17%) ExecOption: Codegen Enabled - BuildTime: 1s674ms - GetNewBlockTime: 69.103us - GetResultsTime: 5.615us - HashBuckets: 6.14K (6144) - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6.81 MB (7144208) - PinTime: 0ns - RowsRepartitioned: 0 - RowsReturned: 6 - RowsReturnedRate: 2.00 /sec - SpilledPartitions: 0 - UnpinTime: 1.898us HDFS_SCAN_NODE (id=0):(Total: 158.846ms, non-child: 158.846ms, % non-child: 100.00%) Hdfs split stats (:<# splits>/): 0:14/3.14 GB Hdfs Read Thread Concurrency Bucket: 0:75% 1:25% 2:0% 3:0% File Formats: PARQUET/SNAPPY:14 ExecOption: Codegen enabled: 0 out of 14 BytesRead(500.0ms): 0, 225.65 MB, 331.54 MB, 331.54 MB, 331.54 MB - AverageHdfsReadThreadConcurrency: 0.25 - AverageScannerThreadConcurrency: 12.75 - BytesRead: 331.54 MB (347643781) - BytesReadDataNodeCache: 0 - BytesReadLocal: 331.54 MB (347643781) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 331.54 MB (347643781) - DecompressionTime: 725.932ms - MaxCompressedTextFileLength: 0 - NumColumns: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 14 - PeakMemoryUsage: 349.28 MB (366250728) - PerReadThreadRawHdfsThroughput: 2.63 GB/sec - RemoteScanRanges: 0 - RowsRead: 46.89M (46887722) - RowsReturned: 46.89M (46887722) - RowsReturnedRate: 295.18 M/sec - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 554 - ScannerThreadsTotalWallClockTime: 24s034ms - MaterializeTupleTime(*): 21s819ms - ScannerThreadsSysTime: 139.469ms - ScannerThreadsUserTime: 2s697ms - ScannerThreadsVoluntaryContextSwitches: 46.98K (46984) - TotalRawHdfsReadTime(*): 123.271ms - TotalReadThroughput: 132.62 MB/sec