================================================================================ SUMMARY: +--------------+--------+----------+---------+----------+-------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +--------------+--------+----------+---------+----------+-------+-----------+ | 03:AGGREGATE | 1 | 43.21ms | 0ns | 43.21ms | 1 | 1 | | 02:EXCHANGE | 1 | 179.55us | 0ns | 179.55us | 15 | 1 | | 01:AGGREGATE | 15 | 231.45ms | 0ns | 256.90ms | 15 | 1 | | 00:SCAN HDFS | 15 | 435.76ms | 0ns | 570.88ms | 1.80B | 180.00M | +--------------+--------+----------+---------+----------+-------+-----------+ PROFILE: Query (id=1e4e9cb4eb284a1f:300eed4151317286): Summary: Session ID: 79441d8f2a1f46e5:7aa13988e7226b9 Session Type: BEESWAX Start Time: 2016-08-21 04:05:46.058046000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.7.0-cdh5-INTERNAL RELEASE (build 7f66659b75a1ad6525d2c630d53b972b81d24ece) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:49152 Default Db: tpch_300_parquet Sql Statement: -- Description : Scan fact table while applying non-selective filter on bigint column. -- Target test case : Basic scan and filter. SELECT count(*) FROM lineitem WHERE l_orderkey > 20 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=98.00MB VCores=1 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=15 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=264 size=64.36GB predicates: l_orderkey > 20 table stats: 1799989091 rows total column stats: all hosts=15 per-host-mem=88.00MB tuple-ids=0 row-size=8B cardinality=179998909 ---------------- Estimated Per-Host Mem: 102760448 Estimated Per-Host VCores: 1 Request Pool: default-pool Planner Timeline: 4.801ms - Analysis finished: 1.494ms (1.494ms) - Equivalence classes computed: 1.663ms (168.174us) - Single node plan created: 2.653ms (990.364us) - Runtime filters computed: 2.688ms (35.096us) - Distributed plan created: 3.122ms (433.427us) - Lineage info computed: 3.699ms (577.315us) - Planning finished: 4.801ms (1.101ms) Query Timeline: 939.487ms - Start execution: 36.596us (36.596us) - Planning finished: 6.804ms (6.768ms) - Ready to start 15 remote fragments: 50.974ms (44.170ms) - All 15 remote fragments started: 227.789ms (176.815ms) - Rows available: 901.191ms (673.401ms) - First row fetched: 934.924ms (33.733ms) - ComputeScanRangeAssignmentTimer: 554.030us ImpalaServer: - ClientFetchWaitTimer: 33.960ms - RowMaterializationTimer: 8.246us Execution Profile 1e4e9cb4eb284a1f:300eed4151317286:(Total: 894.941ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 15, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 84ms, 90th %-ile: 158ms, 95th %-ile: 158ms, 99.9th %-ile: 176ms Per Node Peak Memory Usage: vb0234.halxg.cloudera.com:22000(143.71 MB) vb0218.halxg.cloudera.com:22000(143.65 MB) vb0208.halxg.cloudera.com:22000(137.64 MB) vb0230.halxg.cloudera.com:22000(138.28 MB) vb0216.halxg.cloudera.com:22000(138.97 MB) vb0206.halxg.cloudera.com:22000(143.65 MB) vb0228.halxg.cloudera.com:22000(139.27 MB) vb0220.halxg.cloudera.com:22000(138.34 MB) vb0222.halxg.cloudera.com:22000(143.65 MB) vb0224.halxg.cloudera.com:22000(143.58 MB) vb0226.halxg.cloudera.com:22000(138.29 MB) vb0214.halxg.cloudera.com:22000(132.90 MB) vb0212.halxg.cloudera.com:22000(143.11 MB) vb0232.halxg.cloudera.com:22000(143.65 MB) vb0204.halxg.cloudera.com:22000(143.40 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 703.463ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 12.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 80.02 KB (81936) - PerHostPeakMemUsage: 0 - PrepareTime: 43.056ms - RowsProduced: 0 (0) - TotalCpuTime: 267.380ms - TotalNetworkReceiveTime: 659.962ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 54.980ms, non-child: 54.980ms, % non-child: 100.00%) - CodegenTime: 609.384us - CompileTime: 2.978ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 7 (7) - NumInstructions: 96 (96) - OptimizationTime: 8.042ms - PrepareTime: 43.233ms AGGREGATION_NODE (id=3):(Total: 703.246ms, non-child: 43.214ms, % non-child: 6.14%) ExecOption: Codegen Enabled - BuildTime: 57.596us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 72.00 KB (73728) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 660.032ms, non-child: 179.546us, % non-child: 0.03%) BytesReceived(500.000ms): 0 - BytesReceived: 240.00 B (240) - ConvertRowBatchTime: 27.610us - DeserializeRowBatchTimer: 69.038us - FirstBatchArrivalWaitTime: 386.278ms - PeakMemoryUsage: 0 - RowsReturned: 15 (15) - RowsReturnedRate: 22.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 668.362ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 4.13 GB, max: 4.45 GB, avg: 4.29 GB, stddev: 127.11 MB completion times: min:575.267ms max:836.954ms mean: 683.163ms stddev:79.298ms execution rates: min:5.03 GB/sec max:7.31 GB/sec mean:6.36 GB/sec stddev:748.23 MB/sec num instances: 15 - AverageThreadTokens: 10.17 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.80 MB (147642817) - PerHostPeakMemUsage: 140.80 MB (147644182) - PrepareTime: 44.657ms - RowsProduced: 1 (1) - TotalCpuTime: 6s024ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 285.751us - TotalStorageWaitTime: 238.788ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.276ms, non-child: 57.276ms, % non-child: 100.00%) - CodegenTime: 428.062us - CompileTime: 3.900ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.065ms - PrepareTime: 43.762ms DataStreamSender (dst_id=2):(Total: 75.091us, non-child: 75.091us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 492.99 KB/sec - OverallThroughput: 218.10 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.352us - TransmitDataRPCTime: 35.001us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 667.210ms, non-child: 231.447ms, % non-child: 34.69%) - BuildTime: 148.977ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 343.07 KB (351300) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 435.763ms, non-child: 435.763ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.20 - AverageScannerThreadConcurrency: 9.63 - BytesRead: 183.88 MB (192812683) - BytesReadDataNodeCache: 0 - BytesReadLocal: 183.88 MB (192812683) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 183.88 MB (192812683) - DecompressionTime: 1s055ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.76 MB (147593938) - PerReadThreadRawHdfsThroughput: 1.35 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 120.00M (119999272) - RowsReturned: 120.00M (119999271) - RowsReturnedRate: 281.66 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 621 (621) - ScannerThreadsTotalWallClockTime: 5s580ms - MaterializeTupleTime(*): 3s281ms - ScannerThreadsSysTime: 124.775ms - ScannerThreadsUserTime: 4s742ms - ScannerThreadsVoluntaryContextSwitches: 14.83K (14830) - TotalRawHdfsReadTime(*): 133.873ms - TotalReadThroughput: 244.94 MB/sec Fragment F00: Instance 1e4e9cb4eb284a1f:300eed415131728e (host=vb0222.halxg.cloudera.com:22000):(Total: 822.840ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB MemoryUsage(500.000ms): 142.92 MB, 75.50 MB ThreadUsage(500.000ms): 12, 9 - AverageThreadTokens: 10.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.65 MB (150624024) - PerHostPeakMemUsage: 143.65 MB (150624024) - PrepareTime: 43.907ms - RowsProduced: 1 (1) - TotalCpuTime: 7s674ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 381.118us - TotalStorageWaitTime: 225.574ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.939ms, non-child: 55.939ms, % non-child: 100.00%) - CodegenTime: 430.232us - CompileTime: 3.609ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.783ms - PrepareTime: 43.023ms DataStreamSender (dst_id=2):(Total: 69.374us, non-child: 69.374us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 464.97 KB/sec - OverallThroughput: 225.23 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 18.062us - TransmitDataRPCTime: 33.604us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 821.556ms, non-child: 250.676ms, % non-child: 30.51%) ExecOption: Codegen Enabled - BuildTime: 164.325ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 570.880ms, non-child: 570.880ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB Hdfs Read Thread Concurrency Bucket: 0:50% 1:50% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.61 MB, 169.71 MB - AverageHdfsReadThreadConcurrency: 0.50 - AverageScannerThreadConcurrency: 9.50 - BytesRead: 180.21 MB (188968418) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.21 MB (188968418) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.21 MB (188968418) - DecompressionTime: 1s094ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.61 MB (150587160) - PerReadThreadRawHdfsThroughput: 1.30 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660009) - RowsReturned: 117.66M (117660009) - RowsReturnedRate: 206.10 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 720 (720) - ScannerThreadsTotalWallClockTime: 7s064ms - MaterializeTupleTime(*): 3s384ms - ScannerThreadsSysTime: 205.964ms - ScannerThreadsUserTime: 4s986ms - ScannerThreadsVoluntaryContextSwitches: 38.75K (38751) - TotalRawHdfsReadTime(*): 135.027ms - TotalReadThroughput: 169.71 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317296 (host=vb0234.halxg.cloudera.com:22000):(Total: 804.295ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB MemoryUsage(500.000ms): 143.33 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.71 MB (150689568) - PerHostPeakMemUsage: 143.71 MB (150689568) - PrepareTime: 43.979ms - RowsProduced: 1 (1) - TotalCpuTime: 7s668ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 463.014us - TotalStorageWaitTime: 204.437ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.704ms, non-child: 56.704ms, % non-child: 100.00%) - CodegenTime: 465.170us - CompileTime: 3.968ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.142ms - PrepareTime: 43.052ms DataStreamSender (dst_id=2):(Total: 103.158us, non-child: 103.158us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 273.80 KB/sec - OverallThroughput: 151.47 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.624us - TransmitDataRPCTime: 57.068us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 802.682ms, non-child: 243.525ms, % non-child: 30.34%) ExecOption: Codegen Enabled - BuildTime: 150.974ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 559.157ms, non-child: 559.157ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.22 MB (188973810) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188973810) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188973810) - DecompressionTime: 1s011ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.67 MB (150652704) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660198) - RowsReturned: 117.66M (117660198) - RowsReturnedRate: 210.42 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 746 (746) - ScannerThreadsTotalWallClockTime: 7s055ms - MaterializeTupleTime(*): 3s150ms - ScannerThreadsSysTime: 202.962ms - ScannerThreadsUserTime: 4s693ms - ScannerThreadsVoluntaryContextSwitches: 48.68K (48680) - TotalRawHdfsReadTime(*): 121.070ms - TotalReadThroughput: 233.25 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317295 (host=vb0218.halxg.cloudera.com:22000):(Total: 791.981ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB MemoryUsage(500.000ms): 40.81 MB, 91.24 MB ThreadUsage(500.000ms): 12, 10 - AverageThreadTokens: 11.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.65 MB (150624024) - PerHostPeakMemUsage: 143.65 MB (150624024) - PrepareTime: 44.966ms - RowsProduced: 1 (1) - TotalCpuTime: 7s367ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 289.808us - TotalStorageWaitTime: 227.513ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.840ms, non-child: 57.840ms, % non-child: 100.00%) - CodegenTime: 368.738us - CompileTime: 3.953ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.218ms - PrepareTime: 44.084ms DataStreamSender (dst_id=2):(Total: 70.406us, non-child: 70.406us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 637.08 KB/sec - OverallThroughput: 221.93 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 29.350us - TransmitDataRPCTime: 24.526us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 790.758ms, non-child: 256.904ms, % non-child: 32.49%) ExecOption: Codegen Enabled - BuildTime: 169.695ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 533.854ms, non-child: 533.854ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 800.00 KB, 180.25 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.00 - BytesRead: 180.25 MB (189002112) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.25 MB (189002112) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.25 MB (189002112) - DecompressionTime: 1s164ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.61 MB (150587160) - PerReadThreadRawHdfsThroughput: 1.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659893) - RowsReturned: 117.66M (117659893) - RowsReturnedRate: 220.40 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 692 (692) - ScannerThreadsTotalWallClockTime: 6s789ms - MaterializeTupleTime(*): 3s593ms - ScannerThreadsSysTime: 194.965ms - ScannerThreadsUserTime: 5s185ms - ScannerThreadsVoluntaryContextSwitches: 28.34K (28340) - TotalRawHdfsReadTime(*): 128.926ms - TotalReadThroughput: 180.25 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317294 (host=vb0208.halxg.cloudera.com:22000):(Total: 716.211ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB MemoryUsage(500.000ms): 0, 87.18 MB ThreadUsage(500.000ms): 1, 8 - AverageThreadTokens: 4.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 137.64 MB (144324376) - PerHostPeakMemUsage: 137.64 MB (144324376) - PrepareTime: 43.207ms - RowsProduced: 1 (1) - TotalCpuTime: 6s238ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 312.200us - TotalStorageWaitTime: 233.860ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.184ms, non-child: 55.184ms, % non-child: 100.00%) - CodegenTime: 459.438us - CompileTime: 3.584ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.808ms - PrepareTime: 42.289ms DataStreamSender (dst_id=2):(Total: 87.404us, non-child: 87.404us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 463.59 KB/sec - OverallThroughput: 178.77 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 22.286us - TransmitDataRPCTime: 33.704us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 715.043ms, non-child: 233.772ms, % non-child: 32.69%) ExecOption: Codegen Enabled - BuildTime: 154.055ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 481.271ms, non-child: 481.271ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 187.51 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.00 - BytesRead: 187.51 MB (196617713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 187.51 MB (196617713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 187.51 MB (196617713) - DecompressionTime: 1s099ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 137.57 MB (144254744) - PerReadThreadRawHdfsThroughput: 1.32 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 122.21M (122205961) - RowsReturned: 122.21M (122205961) - RowsReturnedRate: 253.92 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 705 (705) - ScannerThreadsTotalWallClockTime: 5s743ms - MaterializeTupleTime(*): 3s508ms - ScannerThreadsSysTime: 132.975ms - ScannerThreadsUserTime: 5s024ms - ScannerThreadsVoluntaryContextSwitches: 11.71K (11715) - TotalRawHdfsReadTime(*): 138.457ms - TotalReadThroughput: 375.02 MB/sec Instance 1e4e9cb4eb284a1f:300eed415131728d (host=vb0224.halxg.cloudera.com:22000):(Total: 676.735ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB MemoryUsage(500.000ms): 129.00 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.58 MB (150558480) - PerHostPeakMemUsage: 143.58 MB (150558480) - PrepareTime: 43.961ms - RowsProduced: 1 (1) - TotalCpuTime: 6s150ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 249.542us - TotalStorageWaitTime: 275.683ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.664ms, non-child: 56.664ms, % non-child: 100.00%) - CodegenTime: 472.452us - CompileTime: 3.989ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.096ms - PrepareTime: 43.028ms DataStreamSender (dst_id=2):(Total: 79.884us, non-child: 79.884us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 485.04 KB/sec - OverallThroughput: 195.60 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 25.992us - TransmitDataRPCTime: 32.214us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 675.633ms, non-child: 245.526ms, % non-child: 36.34%) ExecOption: Codegen Enabled - BuildTime: 165.342ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 430.106ms, non-child: 430.106ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 190.85 MB (200123337) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200123337) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200123337) - DecompressionTime: 1s132ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.55 MB (150521616) - PerReadThreadRawHdfsThroughput: 1.35 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581098) - RowsReturned: 124.58M (124581098) - RowsReturnedRate: 289.65 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 623 (623) - ScannerThreadsTotalWallClockTime: 5s735ms - MaterializeTupleTime(*): 3s518ms - ScannerThreadsSysTime: 113.978ms - ScannerThreadsUserTime: 5s048ms - ScannerThreadsVoluntaryContextSwitches: 10.01K (10009) - TotalRawHdfsReadTime(*): 138.427ms - TotalReadThroughput: 233.26 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317290 (host=vb0228.halxg.cloudera.com:22000):(Total: 665.697ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB MemoryUsage(500.000ms): 64.55 MB, 46.32 MB ThreadUsage(500.000ms): 12, 6 - AverageThreadTokens: 9.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 139.27 MB (146033968) - PerHostPeakMemUsage: 139.27 MB (146033968) - PrepareTime: 43.930ms - RowsProduced: 1 (1) - TotalCpuTime: 5s696ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 334.862us - TotalStorageWaitTime: 243.495ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.866ms, non-child: 56.866ms, % non-child: 100.00%) - CodegenTime: 371.352us - CompileTime: 3.984ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.213ms - PrepareTime: 43.104ms DataStreamSender (dst_id=2):(Total: 85.914us, non-child: 85.914us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 329.78 KB/sec - OverallThroughput: 181.87 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.252us - TransmitDataRPCTime: 47.380us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 664.437ms, non-child: 230.665ms, % non-child: 34.72%) ExecOption: Codegen Enabled - BuildTime: 148.306ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 433.772ms, non-child: 433.772ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:50% 1:0% 2:0% 3:0% 4:0% 5:50% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 1.07 MB, 178.15 MB - AverageHdfsReadThreadConcurrency: 2.50 - AverageScannerThreadConcurrency: 8.00 - BytesRead: 178.15 MB (186799554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 178.15 MB (186799554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 178.15 MB (186799554) - DecompressionTime: 1s041ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 139.23 MB (145997104) - PerReadThreadRawHdfsThroughput: 1.30 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 116.10M (116100635) - RowsReturned: 116.10M (116100635) - RowsReturnedRate: 267.65 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 601 (601) - ScannerThreadsTotalWallClockTime: 5s260ms - MaterializeTupleTime(*): 3s266ms - ScannerThreadsSysTime: 97.980ms - ScannerThreadsUserTime: 4s711ms - ScannerThreadsVoluntaryContextSwitches: 9.63K (9632) - TotalRawHdfsReadTime(*): 134.215ms - TotalReadThroughput: 178.15 MB/sec Instance 1e4e9cb4eb284a1f:300eed415131728b (host=vb0204.halxg.cloudera.com:22000):(Total: 649.966ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB MemoryUsage(500.000ms): 83.66 MB ThreadUsage(500.000ms): 9 - AverageThreadTokens: 9.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.38 MB (150350064) - PerHostPeakMemUsage: 143.40 MB (150362352) - PrepareTime: 46.817ms - RowsProduced: 1 (1) - TotalCpuTime: 5s696ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 125.084us - TotalStorageWaitTime: 229.999ms CodeGen:(Total: 59.763ms, non-child: 59.763ms, % non-child: 100.00%) - CodegenTime: 378.308us - CompileTime: 4.006ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.215ms - PrepareTime: 45.970ms DataStreamSender (dst_id=2):(Total: 48.080us, non-child: 48.080us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 729.46 KB/sec - OverallThroughput: 324.98 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 13.658us - TransmitDataRPCTime: 21.420us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 649.165ms, non-child: 231.127ms, % non-child: 35.60%) ExecOption: Codegen Enabled - BuildTime: 146.502ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 418.037ms, non-child: 418.037ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 180.24 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 8.00 - BytesRead: 180.24 MB (188998770) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.24 MB (188998770) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.24 MB (188998770) - DecompressionTime: 1s041ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.33 MB (150296816) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659917) - RowsReturned: 117.66M (117659917) - RowsReturnedRate: 281.46 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 532 (532) - ScannerThreadsTotalWallClockTime: 5s262ms - MaterializeTupleTime(*): 3s231ms - ScannerThreadsSysTime: 112.977ms - ScannerThreadsUserTime: 4s646ms - ScannerThreadsVoluntaryContextSwitches: 8.77K (8769) - TotalRawHdfsReadTime(*): 122.990ms - TotalReadThroughput: 360.49 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317288 (host=vb0232.halxg.cloudera.com:22000):(Total: 649.965ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB MemoryUsage(500.000ms): 143.29 MB, 29.15 MB ThreadUsage(500.000ms): 12, 8 - AverageThreadTokens: 10.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.65 MB (150624024) - PerHostPeakMemUsage: 143.65 MB (150624024) - PrepareTime: 49.886ms - RowsProduced: 1 (1) - TotalCpuTime: 5s856ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 294.590us - TotalStorageWaitTime: 246.564ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 62.710ms, non-child: 62.710ms, % non-child: 100.00%) - CodegenTime: 465.968us - CompileTime: 4.080ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.175ms - PrepareTime: 48.905ms DataStreamSender (dst_id=2):(Total: 80.274us, non-child: 80.274us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 478.65 KB/sec - OverallThroughput: 194.65 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 32.796us - TransmitDataRPCTime: 32.644us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 648.766ms, non-child: 236.831ms, % non-child: 36.50%) ExecOption: Codegen Enabled - BuildTime: 151.399ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 411.934ms, non-child: 411.934ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 117.14 MB, 190.57 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 9.00 - BytesRead: 190.57 MB (199830179) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.57 MB (199830179) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.57 MB (199830179) - DecompressionTime: 1s051ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.61 MB (150587160) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.25M (124251724) - RowsReturned: 124.25M (124251724) - RowsReturnedRate: 301.63 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 644 (644) - ScannerThreadsTotalWallClockTime: 5s439ms - MaterializeTupleTime(*): 3s241ms - ScannerThreadsSysTime: 126.973ms - ScannerThreadsUserTime: 4s690ms - ScannerThreadsVoluntaryContextSwitches: 12.57K (12574) - TotalRawHdfsReadTime(*): 139.975ms - TotalReadThroughput: 190.57 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317292 (host=vb0216.halxg.cloudera.com:22000):(Total: 647.662ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB MemoryUsage(500.000ms): 121.03 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.97 MB (145719608) - PerHostPeakMemUsage: 138.97 MB (145719608) - PrepareTime: 43.220ms - RowsProduced: 1 (1) - TotalCpuTime: 5s782ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 194.340us - TotalStorageWaitTime: 233.234ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.947ms, non-child: 55.947ms, % non-child: 100.00%) - CodegenTime: 436.688us - CompileTime: 4.022ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.127ms - PrepareTime: 42.244ms DataStreamSender (dst_id=2):(Total: 72.294us, non-child: 72.294us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 495.53 KB/sec - OverallThroughput: 216.13 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.402us - TransmitDataRPCTime: 31.532us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 646.625ms, non-child: 230.145ms, % non-child: 35.59%) ExecOption: Codegen Enabled - BuildTime: 148.029ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 416.480ms, non-child: 416.480ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189013580) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189013580) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189013580) - DecompressionTime: 1s079ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.91 MB (145658168) - PerReadThreadRawHdfsThroughput: 1.19 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659989) - RowsReturned: 117.66M (117659989) - RowsReturnedRate: 282.51 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 690 (690) - ScannerThreadsTotalWallClockTime: 5s353ms - MaterializeTupleTime(*): 3s330ms - ScannerThreadsSysTime: 104.979ms - ScannerThreadsUserTime: 4s788ms - ScannerThreadsVoluntaryContextSwitches: 8.95K (8951) - TotalRawHdfsReadTime(*): 148.136ms - TotalReadThroughput: 233.25 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317291 (host=vb0206.halxg.cloudera.com:22000):(Total: 645.796ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB MemoryUsage(500.000ms): 0, 87.04 MB ThreadUsage(500.000ms): 1, 8 - AverageThreadTokens: 4.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.65 MB (150624024) - PerHostPeakMemUsage: 143.65 MB (150624024) - PrepareTime: 46.334ms - RowsProduced: 1 (1) - TotalCpuTime: 5s906ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 297.664us - TotalStorageWaitTime: 239.423ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.364ms, non-child: 58.364ms, % non-child: 100.00%) - CodegenTime: 331.774us - CompileTime: 3.478ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.705ms - PrepareTime: 45.673ms DataStreamSender (dst_id=2):(Total: 56.046us, non-child: 56.046us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 575.32 KB/sec - OverallThroughput: 278.79 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 17.138us - TransmitDataRPCTime: 27.159us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 644.616ms, non-child: 233.429ms, % non-child: 36.21%) ExecOption: Codegen Enabled - BuildTime: 146.626ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 411.187ms, non-child: 411.187ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 190.82 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.00 - BytesRead: 190.82 MB (200089526) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.82 MB (200089526) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.82 MB (200089526) - DecompressionTime: 1s099ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.61 MB (150587160) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124580590) - RowsReturned: 124.58M (124580590) - RowsReturnedRate: 302.98 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 601 (601) - ScannerThreadsTotalWallClockTime: 5s486ms - MaterializeTupleTime(*): 3s459ms - ScannerThreadsSysTime: 127.973ms - ScannerThreadsUserTime: 4s898ms - ScannerThreadsVoluntaryContextSwitches: 11.73K (11730) - TotalRawHdfsReadTime(*): 133.015ms - TotalReadThroughput: 381.64 MB/sec Instance 1e4e9cb4eb284a1f:300eed415131728c (host=vb0226.halxg.cloudera.com:22000):(Total: 616.510ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB MemoryUsage(500.000ms): 137.82 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.29 MB (145004192) - PerHostPeakMemUsage: 138.29 MB (145004192) - PrepareTime: 42.823ms - RowsProduced: 1 (1) - TotalCpuTime: 5s576ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 269.016us - TotalStorageWaitTime: 265.901ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.675ms, non-child: 55.675ms, % non-child: 100.00%) - CodegenTime: 466.148us - CompileTime: 4.084ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.178ms - PrepareTime: 41.853ms DataStreamSender (dst_id=2):(Total: 66.378us, non-child: 66.378us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 538.46 KB/sec - OverallThroughput: 235.39 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.176us - TransmitDataRPCTime: 29.018us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 615.398ms, non-child: 233.562ms, % non-child: 37.95%) ExecOption: Codegen Enabled - BuildTime: 153.817ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 381.836ms, non-child: 381.836ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 116.62 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 190.83 MB (200100620) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.83 MB (200100620) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.83 MB (200100620) - DecompressionTime: 1s052ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.22 MB (144934560) - PerReadThreadRawHdfsThroughput: 1.27 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581171) - RowsReturned: 124.58M (124581171) - RowsReturnedRate: 326.27 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 537 (537) - ScannerThreadsTotalWallClockTime: 5s212ms - MaterializeTupleTime(*): 3s278ms - ScannerThreadsSysTime: 94.980ms - ScannerThreadsUserTime: 4s732ms - ScannerThreadsVoluntaryContextSwitches: 7.82K (7819) - TotalRawHdfsReadTime(*): 147.158ms - TotalReadThroughput: 233.24 MB/sec Instance 1e4e9cb4eb284a1f:300eed415131728a (host=vb0214.halxg.cloudera.com:22000):(Total: 606.321ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB MemoryUsage(500.000ms): 117.94 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 132.90 MB (139359904) - PerHostPeakMemUsage: 132.90 MB (139359904) - PrepareTime: 43.607ms - RowsProduced: 1 (1) - TotalCpuTime: 5s354ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 335.676us - TotalStorageWaitTime: 262.377ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.500ms, non-child: 56.500ms, % non-child: 100.00%) - CodegenTime: 479.718us - CompileTime: 4.080ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.206ms - PrepareTime: 42.633ms DataStreamSender (dst_id=2):(Total: 112.790us, non-child: 112.790us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 242.26 KB/sec - OverallThroughput: 138.53 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 20.068us - TransmitDataRPCTime: 64.495us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 605.047ms, non-child: 201.709ms, % non-child: 33.34%) ExecOption: Codegen Enabled - BuildTime: 127.616ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 403.337ms, non-child: 403.337ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 123.96 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 176.97 MB (185568795) - BytesReadDataNodeCache: 0 - BytesReadLocal: 176.97 MB (185568795) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 176.97 MB (185568795) - DecompressionTime: 1s010ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 132.84 MB (139290272) - PerReadThreadRawHdfsThroughput: 1.23 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 115.49M (115486915) - RowsReturned: 115.49M (115486915) - RowsReturnedRate: 286.33 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 567 (567) - ScannerThreadsTotalWallClockTime: 4s996ms - MaterializeTupleTime(*): 3s155ms - ScannerThreadsSysTime: 109.978ms - ScannerThreadsUserTime: 4s553ms - ScannerThreadsVoluntaryContextSwitches: 7.11K (7106) - TotalRawHdfsReadTime(*): 140.232ms - TotalReadThroughput: 247.92 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317293 (host=vb0230.halxg.cloudera.com:22000):(Total: 605.955ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB MemoryUsage(500.000ms): 1020.00 KB, 31.29 MB ThreadUsage(500.000ms): 12, 8 - AverageThreadTokens: 10.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.28 MB (144993888) - PerHostPeakMemUsage: 138.28 MB (144993888) - PrepareTime: 46.926ms - RowsProduced: 1 (1) - TotalCpuTime: 5s386ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 200.582us - TotalStorageWaitTime: 223.868ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 59.819ms, non-child: 59.819ms, % non-child: 100.00%) - CodegenTime: 471.324us - CompileTime: 4.162ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.143ms - PrepareTime: 45.966ms DataStreamSender (dst_id=2):(Total: 58.174us, non-child: 58.174us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 555.02 KB/sec - OverallThroughput: 268.59 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 13.530us - TransmitDataRPCTime: 28.152us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 604.995ms, non-child: 228.682ms, % non-child: 37.80%) ExecOption: Codegen Enabled - BuildTime: 146.454ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 376.313ms, non-child: 376.313ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 0, 190.85 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 9.00 - BytesRead: 190.85 MB (200116100) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200116100) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200116100) - DecompressionTime: 1s029ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.23 MB (144948832) - PerReadThreadRawHdfsThroughput: 1.49 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581188) - RowsReturned: 124.58M (124581188) - RowsReturnedRate: 331.06 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 624 (624) - ScannerThreadsTotalWallClockTime: 4s990ms - MaterializeTupleTime(*): 3s193ms - ScannerThreadsSysTime: 79.983ms - ScannerThreadsUserTime: 4s610ms - ScannerThreadsVoluntaryContextSwitches: 6.77K (6774) - TotalRawHdfsReadTime(*): 125.250ms - TotalReadThroughput: 190.85 MB/sec Instance 1e4e9cb4eb284a1f:300eed415131728f (host=vb0220.halxg.cloudera.com:22000):(Total: 565.237ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB MemoryUsage(500.000ms): 81.36 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.34 MB (145061552) - PerHostPeakMemUsage: 138.34 MB (145061552) - PrepareTime: 42.254ms - RowsProduced: 1 (1) - TotalCpuTime: 5s010ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 311.408us - TotalStorageWaitTime: 246.109ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 54.327ms, non-child: 54.327ms, % non-child: 100.00%) - CodegenTime: 359.610us - CompileTime: 3.479ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.746ms - PrepareTime: 41.567ms DataStreamSender (dst_id=2):(Total: 70.686us, non-child: 70.686us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 379.56 KB/sec - OverallThroughput: 221.05 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 14.684us - TransmitDataRPCTime: 41.166us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 564.188ms, non-child: 205.814ms, % non-child: 36.48%) ExecOption: Codegen Enabled - BuildTime: 130.058ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 379.01 KB (388104) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 358.374ms, non-child: 358.374ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189016856) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189016856) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189016856) - DecompressionTime: 969.423ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.31 MB (145024688) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659978) - RowsReturned: 117.66M (117659953) - RowsReturnedRate: 328.32 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 551 (551) - ScannerThreadsTotalWallClockTime: 4s678ms - MaterializeTupleTime(*): 2s953ms - ScannerThreadsSysTime: 78.982ms - ScannerThreadsUserTime: 4s289ms - ScannerThreadsVoluntaryContextSwitches: 6.09K (6091) - TotalRawHdfsReadTime(*): 132.367ms - TotalReadThroughput: 233.26 MB/sec Instance 1e4e9cb4eb284a1f:300eed4151317289 (host=vb0212.halxg.cloudera.com:22000):(Total: 560.253ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB MemoryUsage(500.000ms): 136.98 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.10 MB (150050560) - PerHostPeakMemUsage: 143.11 MB (150058752) - PrepareTime: 44.042ms - RowsProduced: 1 (1) - TotalCpuTime: 4s996ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 227.362us - TotalStorageWaitTime: 223.782ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.842ms, non-child: 56.842ms, % non-child: 100.00%) - CodegenTime: 464.012us - CompileTime: 4.020ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.224ms - PrepareTime: 43.040ms DataStreamSender (dst_id=2):(Total: 65.507us, non-child: 65.507us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 746.32 KB/sec - OverallThroughput: 238.52 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.270us - TransmitDataRPCTime: 20.936us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 559.242ms, non-child: 209.340ms, % non-child: 37.43%) ExecOption: Codegen Enabled - BuildTime: 131.456ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 349.902ms, non-child: 349.902ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.61 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.22 MB (188970878) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188970878) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188970878) - DecompressionTime: 958.800ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.03 MB (149980928) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659825) - RowsReturned: 117.66M (117659825) - RowsReturnedRate: 336.26 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 489 (489) - ScannerThreadsTotalWallClockTime: 4s645ms - MaterializeTupleTime(*): 2s959ms - ScannerThreadsSysTime: 85.982ms - ScannerThreadsUserTime: 4s281ms - ScannerThreadsVoluntaryContextSwitches: 5.51K (5510) - TotalRawHdfsReadTime(*): 122.857ms - TotalReadThroughput: 233.22 MB/sec ================================================================================ ================================================================================ SUMMARY: +--------------+--------+----------+---------+----------+-------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +--------------+--------+----------+---------+----------+-------+-----------+ | 03:AGGREGATE | 1 | 42.85ms | 0ns | 42.85ms | 1 | 1 | | 02:EXCHANGE | 1 | 186.29us | 0ns | 186.29us | 15 | 1 | | 01:AGGREGATE | 15 | 229.14ms | 0ns | 264.07ms | 15 | 1 | | 00:SCAN HDFS | 15 | 430.90ms | 0ns | 633.54ms | 1.80B | 180.00M | +--------------+--------+----------+---------+----------+-------+-----------+ PROFILE: Query (id=444e35ddbd56350:beefe97c0652f6af): Summary: Session ID: 1b45732c606a1bf0:66b582efdd3f1f86 Session Type: BEESWAX Start Time: 2016-08-21 04:05:47.064021000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.7.0-cdh5-INTERNAL RELEASE (build 7f66659b75a1ad6525d2c630d53b972b81d24ece) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:49154 Default Db: tpch_300_parquet Sql Statement: -- Description : Scan fact table while applying non-selective filter on bigint column. -- Target test case : Basic scan and filter. SELECT count(*) FROM lineitem WHERE l_orderkey > 20 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=98.00MB VCores=1 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=15 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=264 size=64.36GB predicates: l_orderkey > 20 table stats: 1799989091 rows total column stats: all hosts=15 per-host-mem=88.00MB tuple-ids=0 row-size=8B cardinality=179998909 ---------------- Estimated Per-Host Mem: 102760448 Estimated Per-Host VCores: 1 Request Pool: default-pool Planner Timeline: 5.013ms - Analysis finished: 1.480ms (1.480ms) - Equivalence classes computed: 1.594ms (113.488us) - Single node plan created: 2.509ms (914.845us) - Runtime filters computed: 2.544ms (35.465us) - Distributed plan created: 2.998ms (453.576us) - Lineage info computed: 3.877ms (879.122us) - Planning finished: 5.013ms (1.136ms) Query Timeline: 1s017ms - Start execution: 34.914us (34.914us) - Planning finished: 7.006ms (6.971ms) - Ready to start 15 remote fragments: 50.739ms (43.733ms) - All 15 remote fragments started: 54.290ms (3.550ms) - Rows available: 965.016ms (910.726ms) - First row fetched: 1s012ms (47.486ms) - ComputeScanRangeAssignmentTimer: 558.074us ImpalaServer: - ClientFetchWaitTimer: 47.826ms - RowMaterializationTimer: 7.590us Execution Profile 444e35ddbd56350:beefe97c0652f6af:(Total: 959.049ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 15, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms Per Node Peak Memory Usage: vb0234.halxg.cloudera.com:22000(140.67 MB) vb0218.halxg.cloudera.com:22000(143.58 MB) vb0208.halxg.cloudera.com:22000(137.65 MB) vb0230.halxg.cloudera.com:22000(139.44 MB) vb0216.halxg.cloudera.com:22000(138.75 MB) vb0206.halxg.cloudera.com:22000(143.71 MB) vb0228.halxg.cloudera.com:22000(140.00 MB) vb0220.halxg.cloudera.com:22000(140.25 MB) vb0222.halxg.cloudera.com:22000(143.71 MB) vb0224.halxg.cloudera.com:22000(141.42 MB) vb0226.halxg.cloudera.com:22000(138.96 MB) vb0214.halxg.cloudera.com:22000(136.86 MB) vb0212.halxg.cloudera.com:22000(143.65 MB) vb0232.halxg.cloudera.com:22000(138.56 MB) vb0204.halxg.cloudera.com:22000(142.13 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 936.276ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 12.00 KB, 12.00 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 80.02 KB (81936) - PerHostPeakMemUsage: 0 - PrepareTime: 42.664ms - RowsProduced: 0 (0) - TotalCpuTime: 111.638ms - TotalNetworkReceiveTime: 893.132ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 59.272ms, non-child: 59.272ms, % non-child: 100.00%) - CodegenTime: 598.022us - CompileTime: 2.833ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 7 (7) - NumInstructions: 96 (96) - OptimizationTime: 13.293ms - PrepareTime: 42.438ms AGGREGATION_NODE (id=3):(Total: 936.050ms, non-child: 42.845ms, % non-child: 4.58%) ExecOption: Codegen Enabled - BuildTime: 79.782us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 72.00 KB (73728) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 893.205ms, non-child: 186.288us, % non-child: 0.02%) BytesReceived(500.000ms): 0 - BytesReceived: 240.00 B (240) - ConvertRowBatchTime: 27.570us - DeserializeRowBatchTimer: 76.032us - FirstBatchArrivalWaitTime: 554.253ms - PeakMemoryUsage: 0 - RowsReturned: 15 (15) - RowsReturnedRate: 16.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 661.172ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 4.13 GB, max: 4.45 GB, avg: 4.29 GB, stddev: 127.11 MB completion times: min:574.295ms max:913.236ms mean: 675.960ms stddev:98.495ms execution rates: min:4.61 GB/sec max:7.23 GB/sec mean:6.46 GB/sec stddev:835.69 MB/sec num instances: 15 - AverageThreadTokens: 9.83 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.62 MB (147450721) - PerHostPeakMemUsage: 140.62 MB (147452633) - PrepareTime: 44.760ms - RowsProduced: 1 (1) - TotalCpuTime: 5s889ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 315.356us - TotalStorageWaitTime: 207.909ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.298ms, non-child: 57.298ms, % non-child: 100.00%) - CodegenTime: 386.084us - CompileTime: 3.883ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.933ms - PrepareTime: 43.938ms DataStreamSender (dst_id=2):(Total: 64.751us, non-child: 64.751us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 534.04 KB/sec - OverallThroughput: 250.63 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 18.688us - TransmitDataRPCTime: 31.912us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 660.036ms, non-child: 229.137ms, % non-child: 34.72%) - BuildTime: 145.595ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 343.07 KB (351300) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 430.899ms, non-child: 430.899ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 8.83 - BytesRead: 183.88 MB (192812683) - BytesReadDataNodeCache: 0 - BytesReadLocal: 183.88 MB (192812683) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 183.88 MB (192812683) - DecompressionTime: 1s012ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.57 MB (147396380) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 120.00M (119999272) - RowsReturned: 120.00M (119999271) - RowsReturnedRate: 287.70 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 578 (578) - ScannerThreadsTotalWallClockTime: 5s422ms - MaterializeTupleTime(*): 3s120ms - ScannerThreadsSysTime: 121.709ms - ScannerThreadsUserTime: 4s546ms - ScannerThreadsVoluntaryContextSwitches: 17.79K (17792) - TotalRawHdfsReadTime(*): 126.572ms - TotalReadThroughput: 240.30 MB/sec Fragment F00: Instance 444e35ddbd56350:beefe97c0652f6b7 (host=vb0222.halxg.cloudera.com:22000):(Total: 898.813ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB MemoryUsage(500.000ms): 143.11 MB, 55.52 MB ThreadUsage(500.000ms): 12, 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.71 MB (150689568) - PerHostPeakMemUsage: 143.71 MB (150689568) - PrepareTime: 46.019ms - RowsProduced: 1 (1) - TotalCpuTime: 8s690ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 369.228us - TotalStorageWaitTime: 205.621ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.254ms, non-child: 58.254ms, % non-child: 100.00%) - CodegenTime: 343.102us - CompileTime: 3.668ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.739ms - PrepareTime: 45.326ms DataStreamSender (dst_id=2):(Total: 53.688us, non-child: 53.688us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 507.57 KB/sec - OverallThroughput: 291.03 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 11.350us - TransmitDataRPCTime: 30.784us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 897.611ms, non-child: 264.070ms, % non-child: 29.42%) ExecOption: Codegen Enabled - BuildTime: 164.067ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 633.541ms, non-child: 633.541ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.61 MB, 116.61 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.21 MB (188968418) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.21 MB (188968418) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.21 MB (188968418) - DecompressionTime: 964.554ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.67 MB (150652704) - PerReadThreadRawHdfsThroughput: 1.12 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660009) - RowsReturned: 117.66M (117660009) - RowsReturnedRate: 185.72 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 827 (827) - ScannerThreadsTotalWallClockTime: 7s984ms - MaterializeTupleTime(*): 2s950ms - ScannerThreadsSysTime: 275.951ms - ScannerThreadsUserTime: 4s523ms - ScannerThreadsVoluntaryContextSwitches: 86.16K (86159) - TotalRawHdfsReadTime(*): 157.812ms - TotalReadThroughput: 116.61 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b2 (host=vb0212.halxg.cloudera.com:22000):(Total: 891.859ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB MemoryUsage(500.000ms): 138.72 MB, 95.52 MB ThreadUsage(500.000ms): 12, 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.65 MB (150624024) - PerHostPeakMemUsage: 143.65 MB (150624024) - PrepareTime: 46.079ms - RowsProduced: 1 (1) - TotalCpuTime: 8s593ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 378.436us - TotalStorageWaitTime: 180.660ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.695ms, non-child: 58.695ms, % non-child: 100.00%) - CodegenTime: 432.068us - CompileTime: 3.985ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.029ms - PrepareTime: 45.144ms DataStreamSender (dst_id=2):(Total: 73.866us, non-child: 73.866us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 479.56 KB/sec - OverallThroughput: 211.53 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 19.126us - TransmitDataRPCTime: 32.582us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 890.419ms, non-child: 259.388ms, % non-child: 29.13%) ExecOption: Codegen Enabled - BuildTime: 161.942ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 631.031ms, non-child: 631.031ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.61 MB, 169.62 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.22 MB (188970878) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188970878) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188970878) - DecompressionTime: 1s031ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.61 MB (150587160) - PerReadThreadRawHdfsThroughput: 1.55 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659825) - RowsReturned: 117.66M (117659825) - RowsReturnedRate: 186.46 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 787 (787) - ScannerThreadsTotalWallClockTime: 7s868ms - MaterializeTupleTime(*): 3s213ms - ScannerThreadsSysTime: 252.955ms - ScannerThreadsUserTime: 4s867ms - ScannerThreadsVoluntaryContextSwitches: 62.06K (62064) - TotalRawHdfsReadTime(*): 113.179ms - TotalReadThroughput: 169.62 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b3 (host=vb0214.halxg.cloudera.com:22000):(Total: 720.567ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB MemoryUsage(500.000ms): 83.90 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 136.86 MB (143513232) - PerHostPeakMemUsage: 136.86 MB (143513232) - PrepareTime: 39.014ms - RowsProduced: 1 (1) - TotalCpuTime: 6s510ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 362.148us - TotalStorageWaitTime: 207.853ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 49.356ms, non-child: 49.356ms, % non-child: 100.00%) - CodegenTime: 393.668us - CompileTime: 3.056ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 7.597ms - PrepareTime: 38.268ms DataStreamSender (dst_id=2):(Total: 93.452us, non-child: 93.452us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 325.53 KB/sec - OverallThroughput: 167.20 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 34.998us - TransmitDataRPCTime: 47.998us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 719.400ms, non-child: 217.765ms, % non-child: 30.27%) ExecOption: Codegen Enabled - BuildTime: 141.252ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 501.635ms, non-child: 501.635ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 155.77 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 176.97 MB (185568795) - BytesReadDataNodeCache: 0 - BytesReadLocal: 176.97 MB (185568795) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 176.97 MB (185568795) - DecompressionTime: 1s055ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 136.82 MB (143468176) - PerReadThreadRawHdfsThroughput: 1.51 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 115.49M (115486915) - RowsReturned: 115.49M (115486915) - RowsReturnedRate: 230.22 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 658 (658) - ScannerThreadsTotalWallClockTime: 5s987ms - MaterializeTupleTime(*): 3s314ms - ScannerThreadsSysTime: 165.969ms - ScannerThreadsUserTime: 4s844ms - ScannerThreadsVoluntaryContextSwitches: 21.82K (21816) - TotalRawHdfsReadTime(*): 114.643ms - TotalReadThroughput: 311.53 MB/sec Instance 444e35ddbd56350:beefe97c0652f6ba (host=vb0206.halxg.cloudera.com:22000):(Total: 659.400ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB MemoryUsage(500.000ms): 143.22 MB, 47.22 MB ThreadUsage(500.000ms): 12, 8 - AverageThreadTokens: 10.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.71 MB (150689568) - PerHostPeakMemUsage: 143.71 MB (150689568) - PrepareTime: 45.432ms - RowsProduced: 1 (1) - TotalCpuTime: 6s078ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 374.302us - TotalStorageWaitTime: 203.915ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.466ms, non-child: 57.466ms, % non-child: 100.00%) - CodegenTime: 343.458us - CompileTime: 3.488ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.767ms - PrepareTime: 44.711ms DataStreamSender (dst_id=2):(Total: 81.243us, non-child: 81.243us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 315.04 KB/sec - OverallThroughput: 192.32 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 20.614us - TransmitDataRPCTime: 49.596us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 658.221ms, non-child: 246.705ms, % non-child: 37.48%) ExecOption: Codegen Enabled - BuildTime: 162.344ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 411.516ms, non-child: 411.516ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 116.61 MB, 190.82 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 9.00 - BytesRead: 190.82 MB (200089526) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.82 MB (200089526) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.82 MB (200089526) - DecompressionTime: 1s106ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.64 MB (150619936) - PerReadThreadRawHdfsThroughput: 1.32 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124580590) - RowsReturned: 124.58M (124580590) - RowsReturnedRate: 302.74 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 624 (624) - ScannerThreadsTotalWallClockTime: 5s610ms - MaterializeTupleTime(*): 3s355ms - ScannerThreadsSysTime: 115.978ms - ScannerThreadsUserTime: 4s836ms - ScannerThreadsVoluntaryContextSwitches: 12.66K (12661) - TotalRawHdfsReadTime(*): 141.077ms - TotalReadThroughput: 190.82 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b5 (host=vb0226.halxg.cloudera.com:22000):(Total: 649.447ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB MemoryUsage(500.000ms): 137.29 MB, 9.00 MB ThreadUsage(500.000ms): 12, 3 - AverageThreadTokens: 7.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.96 MB (145708696) - PerHostPeakMemUsage: 138.96 MB (145708696) - PrepareTime: 44.206ms - RowsProduced: 1 (1) - TotalCpuTime: 5s407ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 268.954us - TotalStorageWaitTime: 214.638ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.048ms, non-child: 57.048ms, % non-child: 100.00%) - CodegenTime: 350.364us - CompileTime: 4.058ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.028ms - PrepareTime: 43.415ms DataStreamSender (dst_id=2):(Total: 67.308us, non-child: 67.308us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 512.19 KB/sec - OverallThroughput: 232.14 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.534us - TransmitDataRPCTime: 30.506us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 648.311ms, non-child: 231.198ms, % non-child: 35.66%) ExecOption: Codegen Enabled - BuildTime: 148.455ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 417.113ms, non-child: 417.113ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 116.63 MB, 190.83 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.50 - BytesRead: 190.83 MB (200100620) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.83 MB (200100620) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.83 MB (200100620) - DecompressionTime: 1s028ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.92 MB (145671832) - PerReadThreadRawHdfsThroughput: 1.35 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581171) - RowsReturned: 124.58M (124581171) - RowsReturnedRate: 298.67 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 501 (501) - ScannerThreadsTotalWallClockTime: 4s958ms - MaterializeTupleTime(*): 3s174ms - ScannerThreadsSysTime: 89.981ms - ScannerThreadsUserTime: 4s600ms - ScannerThreadsVoluntaryContextSwitches: 6.35K (6346) - TotalRawHdfsReadTime(*): 138.276ms - TotalReadThroughput: 190.83 MB/sec Instance 444e35ddbd56350:beefe97c0652f6bd (host=vb0208.halxg.cloudera.com:22000):(Total: 637.888ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB MemoryUsage(500.000ms): 135.48 MB, 26.89 MB ThreadUsage(500.000ms): 12, 7 - AverageThreadTokens: 9.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 137.65 MB (144332568) - PerHostPeakMemUsage: 137.65 MB (144332568) - PrepareTime: 45.960ms - RowsProduced: 1 (1) - TotalCpuTime: 5s920ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 271.128us - TotalStorageWaitTime: 208.478ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.023ms, non-child: 58.023ms, % non-child: 100.00%) - CodegenTime: 432.424us - CompileTime: 3.665ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.762ms - PrepareTime: 45.084ms DataStreamSender (dst_id=2):(Total: 54.930us, non-child: 54.930us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 598.43 KB/sec - OverallThroughput: 284.45 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 14.328us - TransmitDataRPCTime: 26.110us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 636.792ms, non-child: 239.777ms, % non-child: 37.65%) ExecOption: Codegen Enabled - BuildTime: 155.852ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 397.015ms, non-child: 397.015ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 113.29 MB, 187.51 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 8.50 - BytesRead: 187.51 MB (196617713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 187.51 MB (196617713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 187.51 MB (196617713) - DecompressionTime: 1s035ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 137.61 MB (144295704) - PerReadThreadRawHdfsThroughput: 1.52 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 122.21M (122205961) - RowsReturned: 122.21M (122205961) - RowsReturnedRate: 307.81 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 549 (549) - ScannerThreadsTotalWallClockTime: 5s477ms - MaterializeTupleTime(*): 3s163ms - ScannerThreadsSysTime: 117.976ms - ScannerThreadsUserTime: 4s578ms - ScannerThreadsVoluntaryContextSwitches: 16.98K (16982) - TotalRawHdfsReadTime(*): 120.844ms - TotalReadThroughput: 187.51 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b8 (host=vb0220.halxg.cloudera.com:22000):(Total: 634.935ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB MemoryUsage(500.000ms): 89.21 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.25 MB (147062672) - PerHostPeakMemUsage: 140.25 MB (147062672) - PrepareTime: 46.255ms - RowsProduced: 1 (1) - TotalCpuTime: 5s288ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 319.760us - TotalStorageWaitTime: 197.928ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 59.274ms, non-child: 59.274ms, % non-child: 100.00%) - CodegenTime: 364.558us - CompileTime: 4.129ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.125ms - PrepareTime: 45.442ms DataStreamSender (dst_id=2):(Total: 60.150us, non-child: 60.150us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 793.39 KB/sec - OverallThroughput: 259.77 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 24.222us - TransmitDataRPCTime: 19.694us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 633.763ms, non-child: 218.641ms, % non-child: 34.50%) ExecOption: Codegen Enabled - BuildTime: 138.153ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 379.01 KB (388104) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 415.121ms, non-child: 415.121ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 169.66 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189016856) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189016856) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189016856) - DecompressionTime: 998.262ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.18 MB (146984840) - PerReadThreadRawHdfsThroughput: 1.41 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659978) - RowsReturned: 117.66M (117659953) - RowsReturnedRate: 283.43 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 566 (566) - ScannerThreadsTotalWallClockTime: 4s838ms - MaterializeTupleTime(*): 3s104ms - ScannerThreadsSysTime: 105.979ms - ScannerThreadsUserTime: 4s476ms - ScannerThreadsVoluntaryContextSwitches: 7.54K (7541) - TotalRawHdfsReadTime(*): 124.955ms - TotalReadThroughput: 339.31 MB/sec Instance 444e35ddbd56350:beefe97c0652f6bf (host=vb0234.halxg.cloudera.com:22000):(Total: 633.496ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB MemoryUsage(500.000ms): 97.00 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.67 MB (147506800) - PerHostPeakMemUsage: 140.67 MB (147506800) - PrepareTime: 46.163ms - RowsProduced: 1 (1) - TotalCpuTime: 5s251ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 298.368us - TotalStorageWaitTime: 202.110ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 59.085ms, non-child: 59.085ms, % non-child: 100.00%) - CodegenTime: 373.326us - CompileTime: 4.039ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.129ms - PrepareTime: 45.337ms DataStreamSender (dst_id=2):(Total: 50.660us, non-child: 50.660us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 669.39 KB/sec - OverallThroughput: 308.43 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 14.338us - TransmitDataRPCTime: 23.342us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 632.376ms, non-child: 214.713ms, % non-child: 33.95%) ExecOption: Codegen Enabled - BuildTime: 133.825ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 417.662ms, non-child: 417.662ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 169.62 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.22 MB (188973810) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188973810) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188973810) - DecompressionTime: 998.582ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.63 MB (147461744) - PerReadThreadRawHdfsThroughput: 1.25 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660198) - RowsReturned: 117.66M (117660198) - RowsReturnedRate: 281.71 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 522 (522) - ScannerThreadsTotalWallClockTime: 4s806ms - MaterializeTupleTime(*): 3s080ms - ScannerThreadsSysTime: 71.983ms - ScannerThreadsUserTime: 4s483ms - ScannerThreadsVoluntaryContextSwitches: 5.48K (5477) - TotalRawHdfsReadTime(*): 140.360ms - TotalReadThroughput: 339.23 MB/sec Instance 444e35ddbd56350:beefe97c0652f6bb (host=vb0216.halxg.cloudera.com:22000):(Total: 626.321ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB MemoryUsage(500.000ms): 113.14 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.75 MB (145491416) - PerHostPeakMemUsage: 138.75 MB (145491416) - PrepareTime: 45.228ms - RowsProduced: 1 (1) - TotalCpuTime: 5s394ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 248.788us - TotalStorageWaitTime: 193.172ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.119ms, non-child: 58.119ms, % non-child: 100.00%) - CodegenTime: 365.508us - CompileTime: 4.030ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.145ms - PrepareTime: 44.382ms DataStreamSender (dst_id=2):(Total: 44.356us, non-child: 44.356us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 858.04 KB/sec - OverallThroughput: 352.26 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 13.748us - TransmitDataRPCTime: 18.210us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 625.267ms, non-child: 223.757ms, % non-child: 35.79%) ExecOption: Codegen Enabled - BuildTime: 138.169ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 401.510ms, non-child: 401.510ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189013580) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189013580) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189013580) - DecompressionTime: 1s035ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.70 MB (145438168) - PerReadThreadRawHdfsThroughput: 1.54 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659989) - RowsReturned: 117.66M (117659989) - RowsReturnedRate: 293.04 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 575 (575) - ScannerThreadsTotalWallClockTime: 4s947ms - MaterializeTupleTime(*): 3s172ms - ScannerThreadsSysTime: 90.981ms - ScannerThreadsUserTime: 4s592ms - ScannerThreadsVoluntaryContextSwitches: 7.92K (7920) - TotalRawHdfsReadTime(*): 114.255ms - TotalReadThroughput: 233.26 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b1 (host=vb0232.halxg.cloudera.com:22000):(Total: 615.960ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB MemoryUsage(500.000ms): 81.26 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.56 MB (145290912) - PerHostPeakMemUsage: 138.56 MB (145290912) - PrepareTime: 44.402ms - RowsProduced: 1 (1) - TotalCpuTime: 5s402ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 309.788us - TotalStorageWaitTime: 226.152ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.373ms, non-child: 57.373ms, % non-child: 100.00%) - CodegenTime: 362.172us - CompileTime: 4.109ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.096ms - PrepareTime: 43.604ms DataStreamSender (dst_id=2):(Total: 60.702us, non-child: 60.702us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 629.38 KB/sec - OverallThroughput: 257.40 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.812us - TransmitDataRPCTime: 24.826us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 614.786ms, non-child: 216.057ms, % non-child: 35.14%) ExecOption: Codegen Enabled - BuildTime: 137.723ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 398.729ms, non-child: 398.729ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 117.14 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 190.57 MB (199830179) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.57 MB (199830179) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.57 MB (199830179) - DecompressionTime: 1s024ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.49 MB (145221280) - PerReadThreadRawHdfsThroughput: 1.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.25M (124251724) - RowsReturned: 124.25M (124251724) - RowsReturnedRate: 311.62 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 521 (521) - ScannerThreadsTotalWallClockTime: 4s999ms - MaterializeTupleTime(*): 3s200ms - ScannerThreadsSysTime: 99.979ms - ScannerThreadsUserTime: 4s621ms - ScannerThreadsVoluntaryContextSwitches: 6.89K (6894) - TotalRawHdfsReadTime(*): 135.401ms - TotalReadThroughput: 234.28 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b6 (host=vb0224.halxg.cloudera.com:22000):(Total: 613.609ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB MemoryUsage(500.000ms): 87.25 MB ThreadUsage(500.000ms): 8 - AverageThreadTokens: 8.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 141.42 MB (148286288) - PerHostPeakMemUsage: 141.42 MB (148286288) - PrepareTime: 45.515ms - RowsProduced: 1 (1) - TotalCpuTime: 5s491ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 273.810us - TotalStorageWaitTime: 214.777ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.238ms, non-child: 58.238ms, % non-child: 100.00%) - CodegenTime: 448.588us - CompileTime: 4.035ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.065ms - PrepareTime: 44.574ms DataStreamSender (dst_id=2):(Total: 80.956us, non-child: 80.956us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 334.32 KB/sec - OverallThroughput: 193.01 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 13.086us - TransmitDataRPCTime: 46.736us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 612.591ms, non-child: 225.923ms, % non-child: 36.88%) ExecOption: Codegen Enabled - BuildTime: 145.710ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 386.668ms, non-child: 386.668ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 190.85 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.00 - BytesRead: 190.85 MB (200123337) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200123337) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200123337) - DecompressionTime: 1s052ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 141.35 MB (148216656) - PerReadThreadRawHdfsThroughput: 1.57 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581098) - RowsReturned: 124.58M (124581098) - RowsReturnedRate: 322.19 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 550 (550) - ScannerThreadsTotalWallClockTime: 5s078ms - MaterializeTupleTime(*): 3s236ms - ScannerThreadsSysTime: 97.979ms - ScannerThreadsUserTime: 4s693ms - ScannerThreadsVoluntaryContextSwitches: 7.16K (7161) - TotalRawHdfsReadTime(*): 119.049ms - TotalReadThroughput: 381.70 MB/sec Instance 444e35ddbd56350:beefe97c0652f6bc (host=vb0230.halxg.cloudera.com:22000):(Total: 612.996ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB MemoryUsage(500.000ms): 36.00 KB, 34.87 MB ThreadUsage(500.000ms): 5, 7 - AverageThreadTokens: 6.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 139.44 MB (146208432) - PerHostPeakMemUsage: 139.44 MB (146208432) - PrepareTime: 43.669ms - RowsProduced: 1 (1) - TotalCpuTime: 5s271ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 335.628us - TotalStorageWaitTime: 234.938ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.717ms, non-child: 56.717ms, % non-child: 100.00%) - CodegenTime: 384.346us - CompileTime: 4.119ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.217ms - PrepareTime: 42.808ms DataStreamSender (dst_id=2):(Total: 64.730us, non-child: 64.730us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 421.79 KB/sec - OverallThroughput: 241.39 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 12.970us - TransmitDataRPCTime: 37.044us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 611.769ms, non-child: 249.977ms, % non-child: 40.86%) ExecOption: Codegen Enabled - BuildTime: 164.309ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 361.791ms, non-child: 361.791ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 0, 190.85 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 5.00 - BytesRead: 190.85 MB (200116100) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200116100) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200116100) - DecompressionTime: 993.331ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 139.38 MB (146155184) - PerReadThreadRawHdfsThroughput: 1.65 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581188) - RowsReturned: 124.58M (124581188) - RowsReturnedRate: 344.35 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 533 (533) - ScannerThreadsTotalWallClockTime: 4s879ms - MaterializeTupleTime(*): 3s054ms - ScannerThreadsSysTime: 81.982ms - ScannerThreadsUserTime: 4s403ms - ScannerThreadsVoluntaryContextSwitches: 6.65K (6647) - TotalRawHdfsReadTime(*): 113.094ms - TotalReadThroughput: 190.85 MB/sec Instance 444e35ddbd56350:beefe97c0652f6be (host=vb0218.halxg.cloudera.com:22000):(Total: 582.908ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB MemoryUsage(500.000ms): 48.98 MB, 17.77 MB ThreadUsage(500.000ms): 12, 5 - AverageThreadTokens: 8.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.58 MB (150550288) - PerHostPeakMemUsage: 143.58 MB (150550288) - PrepareTime: 42.616ms - RowsProduced: 1 (1) - TotalCpuTime: 5s044ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 392.076us - TotalStorageWaitTime: 214.549ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.630ms, non-child: 55.630ms, % non-child: 100.00%) - CodegenTime: 354.416us - CompileTime: 4.032ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.227ms - PrepareTime: 41.778ms DataStreamSender (dst_id=2):(Total: 61.268us, non-child: 61.268us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 440.07 KB/sec - OverallThroughput: 255.03 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 16.250us - TransmitDataRPCTime: 35.506us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 581.737ms, non-child: 206.594ms, % non-child: 35.51%) ExecOption: Codegen Enabled - BuildTime: 127.726ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 375.143ms, non-child: 375.143ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 900.00 KB, 180.25 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.50 - BytesRead: 180.25 MB (189002112) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.25 MB (189002112) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.25 MB (189002112) - DecompressionTime: 942.010ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.51 MB (150480656) - PerReadThreadRawHdfsThroughput: 1.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659893) - RowsReturned: 117.66M (117659893) - RowsReturnedRate: 313.64 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 468 (468) - ScannerThreadsTotalWallClockTime: 4s662ms - MaterializeTupleTime(*): 2s868ms - ScannerThreadsSysTime: 81.982ms - ScannerThreadsUserTime: 4s156ms - ScannerThreadsVoluntaryContextSwitches: 9.35K (9350) - TotalRawHdfsReadTime(*): 128.181ms - TotalReadThroughput: 180.25 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b4 (host=vb0204.halxg.cloudera.com:22000):(Total: 580.265ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB MemoryUsage(500.000ms): 74.70 MB ThreadUsage(500.000ms): 7 - AverageThreadTokens: 7.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 142.12 MB (149024480) - PerHostPeakMemUsage: 142.13 MB (149036768) - PrepareTime: 45.217ms - RowsProduced: 1 (1) - TotalCpuTime: 5s020ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 237.154us - TotalStorageWaitTime: 201.524ms CodeGen:(Total: 57.725ms, non-child: 57.725ms, % non-child: 100.00%) - CodegenTime: 383.084us - CompileTime: 3.716ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.951ms - PrepareTime: 44.477ms DataStreamSender (dst_id=2):(Total: 53.652us, non-child: 53.652us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 600.22 KB/sec - OverallThroughput: 291.23 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 15.776us - TransmitDataRPCTime: 26.032us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 579.527ms, non-child: 218.604ms, % non-child: 37.72%) ExecOption: Codegen Enabled - BuildTime: 138.469ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 360.922ms, non-child: 360.922ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 180.24 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 180.24 MB (188998770) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.24 MB (188998770) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.24 MB (188998770) - DecompressionTime: 959.930ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 142.06 MB (148963040) - PerReadThreadRawHdfsThroughput: 1.57 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659917) - RowsReturned: 117.66M (117659917) - RowsReturnedRate: 326.00 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 530 (530) - ScannerThreadsTotalWallClockTime: 4s628ms - MaterializeTupleTime(*): 2s948ms - ScannerThreadsSysTime: 83.982ms - ScannerThreadsUserTime: 4s268ms - ScannerThreadsVoluntaryContextSwitches: 5.36K (5364) - TotalRawHdfsReadTime(*): 112.415ms - TotalReadThroughput: 360.49 MB/sec Instance 444e35ddbd56350:beefe97c0652f6b9 (host=vb0228.halxg.cloudera.com:22000):(Total: 559.111ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB MemoryUsage(500.000ms): 1020.00 KB, 4.47 MB ThreadUsage(500.000ms): 12, 2 - AverageThreadTokens: 7.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 139.98 MB (146781880) - PerHostPeakMemUsage: 140.00 MB (146798264) - PrepareTime: 45.632ms - RowsProduced: 1 (1) - TotalCpuTime: 4s974ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 290.784us - TotalStorageWaitTime: 212.314ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.477ms, non-child: 58.477ms, % non-child: 100.00%) - CodegenTime: 460.186us - CompileTime: 4.111ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.112ms - PrepareTime: 44.713ms DataStreamSender (dst_id=2):(Total: 70.314us, non-child: 70.314us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 525.70 KB/sec - OverallThroughput: 222.22 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 22.168us - TransmitDataRPCTime: 29.722us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 557.972ms, non-child: 203.889ms, % non-child: 36.54%) ExecOption: Codegen Enabled - BuildTime: 125.928ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 354.082ms, non-child: 354.082ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 0, 178.15 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 178.15 MB (186799554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 178.15 MB (186799554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 178.15 MB (186799554) - DecompressionTime: 955.497ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 139.93 MB (146728632) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 116.10M (116100635) - RowsReturned: 116.10M (116100635) - RowsReturnedRate: 327.89 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 464 (464) - ScannerThreadsTotalWallClockTime: 4s613ms - MaterializeTupleTime(*): 2s961ms - ScannerThreadsSysTime: 91.981ms - ScannerThreadsUserTime: 4s260ms - ScannerThreadsVoluntaryContextSwitches: 4.51K (4511) - TotalRawHdfsReadTime(*): 125.040ms - TotalReadThroughput: 178.15 MB/sec ================================================================================ ================================================================================ SUMMARY: +--------------+--------+----------+---------+----------+-------+-----------+ | Operator | #Hosts | Avg Time | Std Dev | Max Time | #Rows | Est #Rows | +--------------+--------+----------+---------+----------+-------+-----------+ | 03:AGGREGATE | 1 | 42.78ms | 0ns | 42.78ms | 1 | 1 | | 02:EXCHANGE | 1 | 133.71us | 0ns | 133.71us | 15 | 1 | | 01:AGGREGATE | 15 | 227.96ms | 0ns | 287.22ms | 15 | 1 | | 00:SCAN HDFS | 15 | 445.08ms | 0ns | 811.84ms | 1.80B | 180.00M | +--------------+--------+----------+---------+----------+-------+-----------+ PROFILE: Query (id=324a26508430f549:7699526a9c937fbe): Summary: Session ID: 5944e92a6b157142:a5b3fc4064069289 Session Type: BEESWAX Start Time: 2016-08-21 04:05:48.148844000 End Time: Query Type: QUERY Query State: CREATED Query Status: OK Impala Version: impalad version 2.7.0-cdh5-INTERNAL RELEASE (build 7f66659b75a1ad6525d2c630d53b972b81d24ece) User: jenkins Connected User: jenkins Delegated User: Network Address: 10.17.193.6:49156 Default Db: tpch_300_parquet Sql Statement: -- Description : Scan fact table while applying non-selective filter on bigint column. -- Target test case : Basic scan and filter. SELECT count(*) FROM lineitem WHERE l_orderkey > 20 Coordinator: vb0204.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=12,SYNC_DDL=1,MAX_NUM_RUNTIME_FILTERS=50 Plan: ---------------- Estimated Per-Host Requirements: Memory=98.00MB VCores=1 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=15 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=15 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=264 size=64.36GB predicates: l_orderkey > 20 table stats: 1799989091 rows total column stats: all hosts=15 per-host-mem=88.00MB tuple-ids=0 row-size=8B cardinality=179998909 ---------------- Estimated Per-Host Mem: 102760448 Estimated Per-Host VCores: 1 Request Pool: default-pool Planner Timeline: 4.638ms - Analysis finished: 1.471ms (1.471ms) - Equivalence classes computed: 1.587ms (116.556us) - Single node plan created: 2.491ms (903.648us) - Runtime filters computed: 2.525ms (34.453us) - Distributed plan created: 2.972ms (446.461us) - Lineage info computed: 3.533ms (561.248us) - Planning finished: 4.638ms (1.105ms) Query Timeline: 1s219ms - Start execution: 35.288us (35.288us) - Planning finished: 6.648ms (6.613ms) - Ready to start 15 remote fragments: 50.407ms (43.758ms) - All 15 remote fragments started: 53.404ms (2.997ms) - Rows available: 1s165ms (1s112ms) - First row fetched: 1s214ms (48.971ms) - ComputeScanRangeAssignmentTimer: 553.138us ImpalaServer: - ClientFetchWaitTimer: 49.248ms - RowMaterializationTimer: 8.294us Execution Profile 324a26508430f549:7699526a9c937fbe:(Total: 1s160ms, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed ------------------------------------------------------------------------------------------------------------------- Fragment start latencies: Count: 15, 25th %-ile: 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 2ms, 95th %-ile: 2ms, 99.9th %-ile: 2ms Per Node Peak Memory Usage: vb0234.halxg.cloudera.com:22000(138.69 MB) vb0218.halxg.cloudera.com:22000(143.31 MB) vb0208.halxg.cloudera.com:22000(137.52 MB) vb0230.halxg.cloudera.com:22000(138.32 MB) vb0216.halxg.cloudera.com:22000(143.58 MB) vb0206.halxg.cloudera.com:22000(138.79 MB) vb0228.halxg.cloudera.com:22000(143.70 MB) vb0220.halxg.cloudera.com:22000(138.41 MB) vb0222.halxg.cloudera.com:22000(138.27 MB) vb0224.halxg.cloudera.com:22000(143.71 MB) vb0226.halxg.cloudera.com:22000(140.65 MB) vb0214.halxg.cloudera.com:22000(133.61 MB) vb0212.halxg.cloudera.com:22000(138.69 MB) vb0232.halxg.cloudera.com:22000(141.58 MB) vb0204.halxg.cloudera.com:22000(142.72 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 1s139ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(500.000ms): 12.00 KB, 76.01 KB - AverageThreadTokens: 0.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 80.02 KB (81936) - PerHostPeakMemUsage: 0 - PrepareTime: 42.710ms - RowsProduced: 0 (0) - TotalCpuTime: 110.607ms - TotalNetworkReceiveTime: 1s096ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.394ms, non-child: 57.394ms, % non-child: 100.00%) - CodegenTime: 626.322us - CompileTime: 2.430ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 7 (7) - NumInstructions: 96 (96) - OptimizationTime: 11.511ms - PrepareTime: 42.719ms AGGREGATION_NODE (id=3):(Total: 1s139ms, non-child: 42.784ms, % non-child: 3.75%) ExecOption: Codegen Enabled - BuildTime: 19.334us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 72.00 KB (73728) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 1s096ms, non-child: 133.708us, % non-child: 0.01%) BytesReceived(500.000ms): 0, 224.00 B - BytesReceived: 240.00 B (240) - ConvertRowBatchTime: 19.890us - DeserializeRowBatchTimer: 62.100us - FirstBatchArrivalWaitTime: 541.677ms - PeakMemoryUsage: 0 - RowsReturned: 15 (15) - RowsReturnedRate: 13.00 /sec - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 674.170ms, non-child: 0.000ns, % non-child: 0.00%) split sizes: min: 4.13 GB, max: 4.45 GB, avg: 4.29 GB, stddev: 127.11 MB completion times: min:558.758ms max:1s114ms mean: 688.859ms stddev:123.394ms execution rates: min:4.00 GB/sec max:7.53 GB/sec mean:6.36 GB/sec stddev:822.89 MB/sec num instances: 15 - AverageThreadTokens: 8.90 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.10 MB (146905963) - PerHostPeakMemUsage: 140.10 MB (146909513) - PrepareTime: 43.649ms - RowsProduced: 1 (1) - TotalCpuTime: 6s094ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 302.057us - TotalStorageWaitTime: 208.390ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.392ms, non-child: 56.392ms, % non-child: 100.00%) - CodegenTime: 403.002us - CompileTime: 3.936ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.112ms - PrepareTime: 42.796ms DataStreamSender (dst_id=2):(Total: 73.047us, non-child: 73.047us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 454.41 KB/sec - OverallThroughput: 218.49 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 20.284us - TransmitDataRPCTime: 36.988us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 673.046ms, non-child: 227.962ms, % non-child: 33.87%) - BuildTime: 146.673ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 343.07 KB (351300) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 445.084ms, non-child: 445.084ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.30 - AverageScannerThreadConcurrency: 8.90 - BytesRead: 183.88 MB (192812683) - BytesReadDataNodeCache: 0 - BytesReadLocal: 183.88 MB (192812683) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 183.88 MB (192812683) - DecompressionTime: 1s047ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.05 MB (146856425) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 120.00M (119999272) - RowsReturned: 120.00M (119999271) - RowsReturnedRate: 279.10 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 607 (607) - ScannerThreadsTotalWallClockTime: 5s615ms - MaterializeTupleTime(*): 3s231ms - ScannerThreadsSysTime: 133.974ms - ScannerThreadsUserTime: 4s697ms - ScannerThreadsVoluntaryContextSwitches: 17.10K (17096) - TotalRawHdfsReadTime(*): 129.958ms - TotalReadThroughput: 298.50 MB/sec Fragment F00: Instance 324a26508430f549:7699526a9c937fc5 (host=vb0224.halxg.cloudera.com:22000):(Total: 1s100ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB MemoryUsage(500.000ms): 143.29 MB, 67.32 MB ThreadUsage(500.000ms): 12, 10 - AverageThreadTokens: 11.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.71 MB (150689568) - PerHostPeakMemUsage: 143.71 MB (150689568) - PrepareTime: 42.522ms - RowsProduced: 1 (1) - TotalCpuTime: 11s353ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 281.848us - TotalStorageWaitTime: 261.093ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.333ms, non-child: 55.333ms, % non-child: 100.00%) - CodegenTime: 354.320us - CompileTime: 3.962ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.144ms - PrepareTime: 41.667ms DataStreamSender (dst_id=2):(Total: 74.714us, non-child: 74.714us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 487.82 KB/sec - OverallThroughput: 209.13 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 31.924us - TransmitDataRPCTime: 32.030us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1s099ms, non-child: 287.221ms, % non-child: 26.13%) ExecOption: Codegen Enabled - BuildTime: 188.438ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 811.841ms, non-child: 811.841ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.37 MB 10:2/506.81 MB 2:2/506.80 MB 0:2/506.80 MB 1:5/1.24 GB 7:3/760.22 MB 3:3/760.18 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 116.63 MB, 169.75 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 10.00 - BytesRead: 190.85 MB (200123337) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200123337) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200123337) - DecompressionTime: 1s184ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.67 MB (150652704) - PerReadThreadRawHdfsThroughput: 1.39 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581098) - RowsReturned: 124.58M (124581098) - RowsReturnedRate: 153.45 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 1.03K (1034) - ScannerThreadsTotalWallClockTime: 10s500ms - MaterializeTupleTime(*): 3s597ms - ScannerThreadsSysTime: 435.928ms - ScannerThreadsUserTime: 5s599ms - ScannerThreadsVoluntaryContextSwitches: 100.23K (100232) - TotalRawHdfsReadTime(*): 134.408ms - TotalReadThroughput: 169.75 MB/sec Instance 324a26508430f549:7699526a9c937fcc (host=vb0208.halxg.cloudera.com:22000):(Total: 766.151ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB MemoryUsage(500.000ms): 0, 76.86 MB ThreadUsage(500.000ms): 1, 12 - AverageThreadTokens: 6.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 137.52 MB (144201464) - PerHostPeakMemUsage: 137.52 MB (144201464) - PrepareTime: 43.859ms - RowsProduced: 1 (1) - TotalCpuTime: 7s254ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 386.166us - TotalStorageWaitTime: 200.866ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.915ms, non-child: 55.915ms, % non-child: 100.00%) - CodegenTime: 447.170us - CompileTime: 3.636ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.801ms - PrepareTime: 42.963ms DataStreamSender (dst_id=2):(Total: 88.304us, non-child: 88.304us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 274.51 KB/sec - OverallThroughput: 176.95 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 12.120us - TransmitDataRPCTime: 56.920us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 764.965ms, non-child: 247.725ms, % non-child: 32.38%) ExecOption: Codegen Enabled - BuildTime: 163.718ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 517.239ms, non-child: 517.239ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 0:1/253.40 MB 9:1/253.39 MB 2:1/253.39 MB 3:1/253.40 MB 6:5/1.24 GB 5:4/1013.59 MB 7:3/486.60 MB 4:1/253.38 MB 10:1/187.22 MB 1:1/253.40 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 155.70 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 187.51 MB (196617713) - BytesReadDataNodeCache: 0 - BytesReadLocal: 187.51 MB (196617713) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 187.51 MB (196617713) - DecompressionTime: 1s164ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 137.49 MB (144164600) - PerReadThreadRawHdfsThroughput: 1.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 122.21M (122205961) - RowsReturned: 122.21M (122205961) - RowsReturnedRate: 236.27 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 712 (712) - ScannerThreadsTotalWallClockTime: 6s676ms - MaterializeTupleTime(*): 3s636ms - ScannerThreadsSysTime: 180.966ms - ScannerThreadsUserTime: 5s281ms - ScannerThreadsVoluntaryContextSwitches: 23.34K (23342) - TotalRawHdfsReadTime(*): 133.904ms - TotalReadThroughput: 311.41 MB/sec Instance 324a26508430f549:7699526a9c937fc6 (host=vb0222.halxg.cloudera.com:22000):(Total: 699.111ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB MemoryUsage(500.000ms): 0, 74.70 MB ThreadUsage(500.000ms): 1, 7 - AverageThreadTokens: 4.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.27 MB (144989360) - PerHostPeakMemUsage: 138.27 MB (144989360) - PrepareTime: 43.853ms - RowsProduced: 1 (1) - TotalCpuTime: 6s252ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 434.592us - TotalStorageWaitTime: 222.400ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.136ms, non-child: 56.136ms, % non-child: 100.00%) - CodegenTime: 449.138us - CompileTime: 3.603ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.010ms - PrepareTime: 43.007ms DataStreamSender (dst_id=2):(Total: 78.334us, non-child: 78.334us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 324.47 KB/sec - OverallThroughput: 199.47 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 12.700us - TransmitDataRPCTime: 48.156us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 697.870ms, non-child: 245.945ms, % non-child: 35.24%) ExecOption: Codegen Enabled - BuildTime: 161.098ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 451.924ms, non-child: 451.924ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 4:1/253.39 MB 8:1/253.41 MB 2:3/760.13 MB 3:2/506.77 MB 7:3/760.22 MB 5:1/253.41 MB 0:2/506.77 MB 1:2/506.81 MB 9:2/506.78 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 180.21 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 180.21 MB (188968418) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.21 MB (188968418) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.21 MB (188968418) - DecompressionTime: 1s065ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.21 MB (144919728) - PerReadThreadRawHdfsThroughput: 1.23 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660009) - RowsReturned: 117.66M (117660009) - RowsReturnedRate: 260.35 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 638 (638) - ScannerThreadsTotalWallClockTime: 5s762ms - MaterializeTupleTime(*): 3s333ms - ScannerThreadsSysTime: 151.971ms - ScannerThreadsUserTime: 4s776ms - ScannerThreadsVoluntaryContextSwitches: 16.15K (16154) - TotalRawHdfsReadTime(*): 143.337ms - TotalReadThroughput: 360.43 MB/sec Instance 324a26508430f549:7699526a9c937fc8 (host=vb0228.halxg.cloudera.com:22000):(Total: 676.638ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB MemoryUsage(500.000ms): 70.73 MB ThreadUsage(500.000ms): 7 - AverageThreadTokens: 7.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.69 MB (150664992) - PerHostPeakMemUsage: 143.70 MB (150681376) - PrepareTime: 43.372ms - RowsProduced: 1 (1) - TotalCpuTime: 6s065ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 266.338us - TotalStorageWaitTime: 196.651ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.215ms, non-child: 56.215ms, % non-child: 100.00%) - CodegenTime: 437.458us - CompileTime: 4.181ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.992ms - PrepareTime: 42.505ms DataStreamSender (dst_id=2):(Total: 74.194us, non-child: 74.194us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 461.89 KB/sec - OverallThroughput: 210.60 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.888us - TransmitDataRPCTime: 33.828us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 675.530ms, non-child: 236.413ms, % non-child: 35.00%) ExecOption: Codegen Enabled - BuildTime: 151.901ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 439.117ms, non-child: 439.117ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:2/445.02 MB 7:1/253.42 MB 6:1/253.39 MB 1:1/253.41 MB 5:3/512.08 MB 9:2/506.78 MB 8:3/760.18 MB 10:3/760.17 MB 2:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 178.15 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 178.15 MB (186799554) - BytesReadDataNodeCache: 0 - BytesReadLocal: 178.15 MB (186799554) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 178.15 MB (186799554) - DecompressionTime: 1s041ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.63 MB (150611744) - PerReadThreadRawHdfsThroughput: 1.45 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 116.10M (116100635) - RowsReturned: 116.10M (116100635) - RowsReturnedRate: 264.40 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 577 (577) - ScannerThreadsTotalWallClockTime: 5s571ms - MaterializeTupleTime(*): 3s154ms - ScannerThreadsSysTime: 149.972ms - ScannerThreadsUserTime: 4s571ms - ScannerThreadsVoluntaryContextSwitches: 18.22K (18216) - TotalRawHdfsReadTime(*): 119.605ms - TotalReadThroughput: 356.29 MB/sec Instance 324a26508430f549:7699526a9c937fc2 (host=vb0214.halxg.cloudera.com:22000):(Total: 676.592ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB MemoryUsage(500.000ms): 89.51 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 133.61 MB (140097184) - PerHostPeakMemUsage: 133.61 MB (140097184) - PrepareTime: 42.486ms - RowsProduced: 1 (1) - TotalCpuTime: 6s067ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 343.602us - TotalStorageWaitTime: 197.537ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.369ms, non-child: 55.369ms, % non-child: 100.00%) - CodegenTime: 470.694us - CompileTime: 4.040ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.287ms - PrepareTime: 41.491ms DataStreamSender (dst_id=2):(Total: 72.936us, non-child: 72.936us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 520.21 KB/sec - OverallThroughput: 214.23 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.808us - TransmitDataRPCTime: 30.036us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 675.453ms, non-child: 225.739ms, % non-child: 33.42%) ExecOption: Codegen Enabled - BuildTime: 146.515ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 449.714ms, non-child: 449.714ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.41 MB 0:1/253.42 MB 1:3/760.14 MB 5:1/253.38 MB 3:4/1013.58 MB 6:1/227.66 MB 7:4/959.92 MB 4:2/506.77 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 166.37 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 176.97 MB (185568795) - BytesReadDataNodeCache: 0 - BytesReadLocal: 176.97 MB (185568795) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 176.97 MB (185568795) - DecompressionTime: 1s043ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 133.54 MB (140027552) - PerReadThreadRawHdfsThroughput: 1.50 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 115.49M (115486915) - RowsReturned: 115.49M (115486915) - RowsReturnedRate: 256.80 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 621 (621) - ScannerThreadsTotalWallClockTime: 5s574ms - MaterializeTupleTime(*): 3s197ms - ScannerThreadsSysTime: 105.979ms - ScannerThreadsUserTime: 4s653ms - ScannerThreadsVoluntaryContextSwitches: 17.11K (17106) - TotalRawHdfsReadTime(*): 115.029ms - TotalReadThroughput: 332.73 MB/sec Instance 324a26508430f549:7699526a9c937fca (host=vb0216.halxg.cloudera.com:22000):(Total: 657.062ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB MemoryUsage(500.000ms): 137.04 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.58 MB (150558480) - PerHostPeakMemUsage: 143.58 MB (150558480) - PrepareTime: 42.312ms - RowsProduced: 1 (1) - TotalCpuTime: 5s661ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 414.056us - TotalStorageWaitTime: 205.973ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.773ms, non-child: 55.773ms, % non-child: 100.00%) - CodegenTime: 345.062us - CompileTime: 3.993ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.710ms - PrepareTime: 41.527ms DataStreamSender (dst_id=2):(Total: 67.030us, non-child: 67.030us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 488.49 KB/sec - OverallThroughput: 233.10 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 13.776us - TransmitDataRPCTime: 31.986us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 655.759ms, non-child: 219.568ms, % non-child: 33.48%) ExecOption: Codegen Enabled - BuildTime: 140.422ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 436.191ms, non-child: 436.191ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 5:1/253.40 MB 9:1/253.39 MB 3:2/506.80 MB 4:3/760.21 MB 2:1/253.39 MB 7:1/253.39 MB 6:3/760.20 MB 0:2/506.79 MB 10:3/760.20 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.64 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189013580) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189013580) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189013580) - DecompressionTime: 1s057ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.55 MB (150521616) - PerReadThreadRawHdfsThroughput: 1.30 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659989) - RowsReturned: 117.66M (117659989) - RowsReturnedRate: 269.74 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 569 (569) - ScannerThreadsTotalWallClockTime: 5s196ms - MaterializeTupleTime(*): 3s260ms - ScannerThreadsSysTime: 101.980ms - ScannerThreadsUserTime: 4s708ms - ScannerThreadsVoluntaryContextSwitches: 9.82K (9820) - TotalRawHdfsReadTime(*): 135.501ms - TotalReadThroughput: 233.28 MB/sec Instance 324a26508430f549:7699526a9c937fc0 (host=vb0232.halxg.cloudera.com:22000):(Total: 636.939ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB MemoryUsage(500.000ms): 135.58 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 141.58 MB (148461328) - PerHostPeakMemUsage: 141.58 MB (148461328) - PrepareTime: 42.065ms - RowsProduced: 1 (1) - TotalCpuTime: 5s670ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 405.406us - TotalStorageWaitTime: 192.626ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.021ms, non-child: 55.021ms, % non-child: 100.00%) - CodegenTime: 351.108us - CompileTime: 4.074ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.107ms - PrepareTime: 41.288ms DataStreamSender (dst_id=2):(Total: 88.538us, non-child: 88.538us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 273.01 KB/sec - OverallThroughput: 176.48 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 14.588us - TransmitDataRPCTime: 57.232us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 635.643ms, non-child: 226.535ms, % non-child: 35.64%) ExecOption: Codegen Enabled - BuildTime: 150.338ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 409.107ms, non-child: 409.107ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 19 Hdfs split stats (:<# splits>/): 4:1/253.38 MB 10:1/7.47 MB 9:4/994.52 MB 6:2/506.80 MB 3:2/506.78 MB 1:2/506.78 MB 7:3/760.15 MB 0:1/253.40 MB 5:3/760.19 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:19 BytesRead(500.000ms): 116.35 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 190.57 MB (199830179) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.57 MB (199830179) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.57 MB (199830179) - DecompressionTime: 1s065ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 19 (19) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 141.55 MB (148424464) - PerReadThreadRawHdfsThroughput: 1.62 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.25M (124251724) - RowsReturned: 124.25M (124251724) - RowsReturnedRate: 303.71 M/sec - ScanRangesComplete: 19 (19) - ScannerThreadsInvoluntaryContextSwitches: 562 (562) - ScannerThreadsTotalWallClockTime: 5s212ms - MaterializeTupleTime(*): 3s283ms - ScannerThreadsSysTime: 99.980ms - ScannerThreadsUserTime: 4s731ms - ScannerThreadsVoluntaryContextSwitches: 7.67K (7670) - TotalRawHdfsReadTime(*): 115.222ms - TotalReadThroughput: 232.71 MB/sec Instance 324a26508430f549:7699526a9c937fc4 (host=vb0226.halxg.cloudera.com:22000):(Total: 634.564ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB MemoryUsage(500.000ms): 72.55 MB, 58.76 MB ThreadUsage(500.000ms): 12, 7 - AverageThreadTokens: 9.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 140.65 MB (147483288) - PerHostPeakMemUsage: 140.65 MB (147483288) - PrepareTime: 42.623ms - RowsProduced: 1 (1) - TotalCpuTime: 5s861ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 293.504us - TotalStorageWaitTime: 209.883ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.199ms, non-child: 55.199ms, % non-child: 100.00%) - CodegenTime: 450.738us - CompileTime: 3.958ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.036ms - PrepareTime: 41.652ms DataStreamSender (dst_id=2):(Total: 82.646us, non-child: 82.646us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 345.49 KB/sec - OverallThroughput: 189.06 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.698us - TransmitDataRPCTime: 45.226us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 633.436ms, non-child: 219.275ms, % non-child: 34.62%) ExecOption: Codegen Enabled - BuildTime: 140.258ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 414.161ms, non-child: 414.161ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 5:1/253.38 MB 1:1/253.42 MB 4:2/506.80 MB 8:3/760.22 MB 7:1/253.41 MB 3:2/506.77 MB 2:3/760.15 MB 10:1/253.40 MB 9:2/506.79 MB 6:2/506.82 MB Hdfs Read Thread Concurrency Bucket: 0:50% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:50% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 1.07 MB, 190.83 MB - AverageHdfsReadThreadConcurrency: 4.50 - AverageScannerThreadConcurrency: 8.50 - BytesRead: 190.83 MB (200100620) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.83 MB (200100620) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.83 MB (200100620) - DecompressionTime: 1s013ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 140.62 MB (147446424) - PerReadThreadRawHdfsThroughput: 1.33 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581171) - RowsReturned: 124.58M (124581171) - RowsReturnedRate: 300.80 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 548 (548) - ScannerThreadsTotalWallClockTime: 5s423ms - MaterializeTupleTime(*): 3s108ms - ScannerThreadsSysTime: 155.971ms - ScannerThreadsUserTime: 4s525ms - ScannerThreadsVoluntaryContextSwitches: 18.21K (18213) - TotalRawHdfsReadTime(*): 139.964ms - TotalReadThroughput: 190.83 MB/sec Instance 324a26508430f549:7699526a9c937fcb (host=vb0230.halxg.cloudera.com:22000):(Total: 631.240ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB MemoryUsage(500.000ms): 87.39 MB ThreadUsage(500.000ms): 8 - AverageThreadTokens: 8.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.31 MB (145028776) - PerHostPeakMemUsage: 138.32 MB (145036968) - PrepareTime: 46.553ms - RowsProduced: 1 (1) - TotalCpuTime: 5s539ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 332.096us - TotalStorageWaitTime: 217.664ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 59.431ms, non-child: 59.431ms, % non-child: 100.00%) - CodegenTime: 489.240us - CompileTime: 4.113ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.183ms - PrepareTime: 45.585ms DataStreamSender (dst_id=2):(Total: 80.896us, non-child: 80.896us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 386.99 KB/sec - OverallThroughput: 193.15 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.836us - TransmitDataRPCTime: 40.376us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 630.049ms, non-child: 217.670ms, % non-child: 34.55%) ExecOption: Codegen Enabled - BuildTime: 136.542ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 412.378ms, non-child: 412.378ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 9:1/253.39 MB 8:3/760.17 MB 5:1/253.38 MB 10:3/760.15 MB 2:1/253.40 MB 4:7/1.73 GB 1:2/506.79 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 190.85 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.00 - BytesRead: 190.85 MB (200116100) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.85 MB (200116100) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.85 MB (200116100) - DecompressionTime: 1s059ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 7 (7) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.27 MB (144983720) - PerReadThreadRawHdfsThroughput: 1.63 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124581188) - RowsReturned: 124.58M (124581188) - RowsReturnedRate: 302.10 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 558 (558) - ScannerThreadsTotalWallClockTime: 5s111ms - MaterializeTupleTime(*): 3s283ms - ScannerThreadsSysTime: 81.982ms - ScannerThreadsUserTime: 4s738ms - ScannerThreadsVoluntaryContextSwitches: 6.64K (6642) - TotalRawHdfsReadTime(*): 114.677ms - TotalReadThroughput: 381.69 MB/sec Instance 324a26508430f549:7699526a9c937fc3 (host=vb0204.halxg.cloudera.com:22000):(Total: 626.041ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB MemoryUsage(500.000ms): 89.09 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 142.71 MB (149643752) - PerHostPeakMemUsage: 142.72 MB (149656040) - PrepareTime: 43.512ms - RowsProduced: 1 (1) - TotalCpuTime: 5s373ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 172.990us - TotalStorageWaitTime: 182.956ms CodeGen:(Total: 56.415ms, non-child: 56.415ms, % non-child: 100.00%) - CodegenTime: 381.540us - CompileTime: 3.712ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.343ms - PrepareTime: 42.780ms DataStreamSender (dst_id=2):(Total: 61.072us, non-child: 61.072us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 607.27 KB/sec - OverallThroughput: 255.84 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 22.380us - TransmitDataRPCTime: 25.730us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 625.175ms, non-child: 213.841ms, % non-child: 34.20%) ExecOption: Codegen Enabled - BuildTime: 133.268ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 411.334ms, non-child: 411.334ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 8:2/506.79 MB 1:1/253.37 MB 5:3/760.22 MB 2:2/506.79 MB 7:1/253.39 MB 4:3/760.18 MB 6:3/760.22 MB 0:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 169.64 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.24 MB (188998770) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.24 MB (188998770) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.24 MB (188998770) - DecompressionTime: 1s003ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 142.68 MB (149606888) - PerReadThreadRawHdfsThroughput: 1.51 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659917) - RowsReturned: 117.66M (117659917) - RowsReturnedRate: 286.04 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 534 (534) - ScannerThreadsTotalWallClockTime: 4s916ms - MaterializeTupleTime(*): 3s119ms - ScannerThreadsSysTime: 94.980ms - ScannerThreadsUserTime: 4s497ms - ScannerThreadsVoluntaryContextSwitches: 6.80K (6805) - TotalRawHdfsReadTime(*): 116.232ms - TotalReadThroughput: 339.29 MB/sec Instance 324a26508430f549:7699526a9c937fcd (host=vb0218.halxg.cloudera.com:22000):(Total: 622.530ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB MemoryUsage(500.000ms): 74.61 MB ThreadUsage(500.000ms): 7 - AverageThreadTokens: 7.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 143.31 MB (150271728) - PerHostPeakMemUsage: 143.31 MB (150271728) - PrepareTime: 45.225ms - RowsProduced: 1 (1) - TotalCpuTime: 5s387ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 256.190us - TotalStorageWaitTime: 197.164ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.956ms, non-child: 57.956ms, % non-child: 100.00%) - CodegenTime: 441.786us - CompileTime: 3.968ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.133ms - PrepareTime: 44.302ms DataStreamSender (dst_id=2):(Total: 83.222us, non-child: 83.222us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 355.45 KB/sec - OverallThroughput: 187.75 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 22.496us - TransmitDataRPCTime: 43.958us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 621.386ms, non-child: 219.540ms, % non-child: 35.33%) ExecOption: Codegen Enabled - BuildTime: 141.013ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 401.846ms, non-child: 401.846ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 10:1/253.39 MB 5:2/506.80 MB 8:1/253.40 MB 9:1/253.38 MB 2:2/506.79 MB 6:3/760.19 MB 7:2/506.77 MB 0:4/1013.57 MB 1:1/253.39 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 180.25 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 180.25 MB (189002112) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.25 MB (189002112) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.25 MB (189002112) - DecompressionTime: 1s026ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 9 (9) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 143.26 MB (150216776) - PerReadThreadRawHdfsThroughput: 1.40 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659893) - RowsReturned: 117.66M (117659893) - RowsReturnedRate: 292.80 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 609 (609) - ScannerThreadsTotalWallClockTime: 4s948ms - MaterializeTupleTime(*): 3s166ms - ScannerThreadsSysTime: 91.980ms - ScannerThreadsUserTime: 4s554ms - ScannerThreadsVoluntaryContextSwitches: 6.35K (6349) - TotalRawHdfsReadTime(*): 125.996ms - TotalReadThroughput: 360.49 MB/sec Instance 324a26508430f549:7699526a9c937fc7 (host=vb0220.halxg.cloudera.com:22000):(Total: 618.491ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB MemoryUsage(500.000ms): 113.00 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.41 MB (145135264) - PerHostPeakMemUsage: 138.41 MB (145135264) - PrepareTime: 44.589ms - RowsProduced: 1 (1) - TotalCpuTime: 5s383ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 193.704us - TotalStorageWaitTime: 221.047ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.824ms, non-child: 57.824ms, % non-child: 100.00%) - CodegenTime: 359.860us - CompileTime: 4.373ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.094ms - PrepareTime: 43.802ms DataStreamSender (dst_id=2):(Total: 61.124us, non-child: 61.124us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 545.49 KB/sec - OverallThroughput: 255.63 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.300us - TransmitDataRPCTime: 28.644us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 617.501ms, non-child: 217.331ms, % non-child: 35.20%) ExecOption: Codegen Enabled - BuildTime: 138.348ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 379.01 KB (388104) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 400.170ms, non-child: 400.170ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 10:2/506.79 MB 6:1/253.39 MB 4:2/506.78 MB 2:2/506.80 MB 5:2/506.78 MB 0:4/1013.57 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.63 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.26 MB (189016856) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.26 MB (189016856) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.26 MB (189016856) - DecompressionTime: 1s001ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.36 MB (145082016) - PerReadThreadRawHdfsThroughput: 1.27 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659978) - RowsReturned: 117.66M (117659953) - RowsReturnedRate: 294.02 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 558 (558) - ScannerThreadsTotalWallClockTime: 4s971ms - MaterializeTupleTime(*): 3s124ms - ScannerThreadsSysTime: 94.980ms - ScannerThreadsUserTime: 4s529ms - ScannerThreadsVoluntaryContextSwitches: 9.45K (9446) - TotalRawHdfsReadTime(*): 138.332ms - TotalReadThroughput: 233.26 MB/sec Instance 324a26508430f549:7699526a9c937fce (host=vb0234.halxg.cloudera.com:22000):(Total: 613.067ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB MemoryUsage(500.000ms): 113.25 MB ThreadUsage(500.000ms): 12 - AverageThreadTokens: 12.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.69 MB (145421976) - PerHostPeakMemUsage: 138.69 MB (145430168) - PrepareTime: 44.336ms - RowsProduced: 1 (1) - TotalCpuTime: 5s167ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 317.174us - TotalStorageWaitTime: 177.371ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 57.118ms, non-child: 57.118ms, % non-child: 100.00%) - CodegenTime: 365.526us - CompileTime: 3.949ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.088ms - PrepareTime: 43.480ms DataStreamSender (dst_id=2):(Total: 60.084us, non-child: 60.084us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 591.23 KB/sec - OverallThroughput: 260.05 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 23.406us - TransmitDataRPCTime: 26.428us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 611.879ms, non-child: 215.977ms, % non-child: 35.30%) ExecOption: Codegen Enabled - BuildTime: 137.707ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 395.901ms, non-child: 395.901ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 6:1/253.39 MB 10:1/253.38 MB 9:2/506.79 MB 7:2/506.79 MB 8:1/253.39 MB 3:2/506.77 MB 5:1/253.40 MB 1:3/760.22 MB 4:3/760.15 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 116.62 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 11.00 - BytesRead: 180.22 MB (188973810) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188973810) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188973810) - DecompressionTime: 985.360ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.63 MB (145360536) - PerReadThreadRawHdfsThroughput: 1.43 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117660198) - RowsReturned: 117.66M (117660198) - RowsReturnedRate: 297.20 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 516 (516) - ScannerThreadsTotalWallClockTime: 4s717ms - MaterializeTupleTime(*): 3s055ms - ScannerThreadsSysTime: 81.983ms - ScannerThreadsUserTime: 4s406ms - ScannerThreadsVoluntaryContextSwitches: 4.71K (4710) - TotalRawHdfsReadTime(*): 122.791ms - TotalReadThroughput: 233.24 MB/sec Instance 324a26508430f549:7699526a9c937fc9 (host=vb0206.halxg.cloudera.com:22000):(Total: 609.911ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB MemoryUsage(500.000ms): 0, 79.08 MB ThreadUsage(500.000ms): 1, 8 - AverageThreadTokens: 4.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.78 MB (145520296) - PerHostPeakMemUsage: 138.79 MB (145528488) - PrepareTime: 43.513ms - RowsProduced: 1 (1) - TotalCpuTime: 5s434ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 198.949us - TotalStorageWaitTime: 252.696ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 55.458ms, non-child: 55.458ms, % non-child: 100.00%) - CodegenTime: 348.761us - CompileTime: 3.460ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 8.665ms - PrepareTime: 42.821ms DataStreamSender (dst_id=2):(Total: 54.566us, non-child: 54.566us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 601.98 KB/sec - OverallThroughput: 286.35 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 16.831us - TransmitDataRPCTime: 25.956us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 608.954ms, non-child: 226.885ms, % non-child: 37.26%) ExecOption: Codegen Enabled - BuildTime: 147.410ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 382.068ms, non-child: 382.068ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 18 Hdfs split stats (:<# splits>/): 0:1/253.39 MB 5:2/506.79 MB 4:2/506.78 MB 8:1/253.41 MB 3:1/253.40 MB 7:4/1013.58 MB 9:3/760.16 MB 10:1/253.41 MB 6:1/253.40 MB 2:2/506.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:18 BytesRead(500.000ms): 190.82 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 7.00 - BytesRead: 190.82 MB (200089526) - BytesReadDataNodeCache: 0 - BytesReadLocal: 190.82 MB (200089526) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 190.82 MB (200089526) - DecompressionTime: 1s047ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 10 (10) - NumRowGroups: 18 (18) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.74 MB (145475240) - PerReadThreadRawHdfsThroughput: 1.07 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 124.58M (124580590) - RowsReturned: 124.58M (124580590) - RowsReturnedRate: 326.07 M/sec - ScanRangesComplete: 18 (18) - ScannerThreadsInvoluntaryContextSwitches: 612 (612) - ScannerThreadsTotalWallClockTime: 5s064ms - MaterializeTupleTime(*): 3s197ms - ScannerThreadsSysTime: 104.977ms - ScannerThreadsUserTime: 4s632ms - ScannerThreadsVoluntaryContextSwitches: 6.15K (6148) - TotalRawHdfsReadTime(*): 174.951ms - TotalReadThroughput: 381.64 MB/sec Instance 324a26508430f549:7699526a9c937fc1 (host=vb0212.halxg.cloudera.com:22000):(Total: 544.062ms, non-child: 0.000ns, % non-child: 0.00%) Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB MemoryUsage(500.000ms): 0, 26.74 MB ThreadUsage(500.000ms): 1, 7 - AverageThreadTokens: 4.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 138.69 MB (145422000) - PerHostPeakMemUsage: 138.69 MB (145422000) - PrepareTime: 43.911ms - RowsProduced: 1 (1) - TotalCpuTime: 4s950ms - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 234.244us - TotalStorageWaitTime: 189.917ms BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 80.00 GB (85899345920) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 56.725ms, non-child: 56.725ms, % non-child: 100.00%) - CodegenTime: 352.640us - CompileTime: 4.015ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.86 MB (1945588) - NumFunctions: 10 (10) - NumInstructions: 144 (144) - OptimizationTime: 9.080ms - PrepareTime: 43.071ms DataStreamSender (dst_id=2):(Total: 68.058us, non-child: 68.058us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 551.85 KB/sec - OverallThroughput: 229.58 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.520us - TransmitDataRPCTime: 28.314us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 543.031ms, non-child: 199.759ms, % non-child: 36.79%) ExecOption: Codegen Enabled - BuildTime: 123.116ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 340.50 KB (348672) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 1.00 /sec - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 343.271ms, non-child: 343.271ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 7:1/253.39 MB 1:3/760.20 MB 6:2/506.80 MB 0:1/253.40 MB 8:4/1013.56 MB 5:1/253.39 MB 4:3/760.20 MB 3:2/506.80 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% File Formats: PARQUET/SNAPPY:17 BytesRead(500.000ms): 180.22 MB - AverageHdfsReadThreadConcurrency: 0.00 - AverageScannerThreadConcurrency: 6.00 - BytesRead: 180.22 MB (188970878) - BytesReadDataNodeCache: 0 - BytesReadLocal: 180.22 MB (188970878) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 180.22 MB (188970878) - DecompressionTime: 952.865ms - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 17 (17) - NumScannerThreadsStarted: 11 (11) - PeakMemoryUsage: 138.62 MB (145352368) - PerReadThreadRawHdfsThroughput: 1.47 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 117.66M (117659825) - RowsReturned: 117.66M (117659825) - RowsReturnedRate: 342.76 M/sec - ScanRangesComplete: 17 (17) - ScannerThreadsInvoluntaryContextSwitches: 463 (463) - ScannerThreadsTotalWallClockTime: 4s582ms - MaterializeTupleTime(*): 2s949ms - ScannerThreadsSysTime: 75.982ms - ScannerThreadsUserTime: 4s257ms - ScannerThreadsVoluntaryContextSwitches: 5.59K (5589) - TotalRawHdfsReadTime(*): 119.423ms - TotalReadThroughput: 360.43 MB/sec ================================================================================