impalad /backends /catalog /hadoop-varz /logs /memz /metrics /queries /rpcz /sessions /threadz /varz Query 3f4226c6295a7e0e:527eeda200000000 Plan Query Text plan Summary Profile Memory Download Thrift Profile Query (id=3f4226c6295a7e0e:527eeda200000000): Summary: Session ID: 4346ff4a893d5c38:27674675746d809f Session Type: BEESWAX Start Time: 2016-09-17 10:41:00.621589000 End Time: 2016-09-17 11:17:53.610627000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.8.0-cdh5-INTERNAL DEBUG (build 471d68b76d0c603f2ac0a5184e93395137e7a068) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: 10.17.234.19:56645 Default Db: tpcds_1000_parquet Sql Statement: select count(*) from store_sales_100kp where ss_item_sk > 10 Coordinator: vd1309.halxg.cloudera.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=26.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. tpcds_1000_parquet.store_sales_100kp 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=7 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=7 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=7 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpcds_1000_parquet.store_sales_100kp, RANDOM] partitions=747457/747457 files=847857 size=153.96GB predicates: ss_item_sk > 10 table stats: unavailable column stats: unavailable hosts=7 per-host-mem=16.00MB tuple-ids=0 row-size=8B cardinality=17409513 ---------------- Estimated Per-Host Mem: 27262976 Estimated Per-Host VCores: 1 Tables Missing Stats: tpcds_1000_parquet.store_sales_100kp Request Pool: root.mmokhtar Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------- 03:AGGREGATE 0 0.000ns 0.000ns 0 1 0 -1.00 B FINALIZE 02:EXCHANGE 0 0.000ns 0.000ns 0 1 0 -1.00 B UNPARTITIONED 01:AGGREGATE 7 3s606ms 3s673ms 7 1 44.00 KB 10.00 MB 00:SCAN HDFS 7 1m3s 1m4s 2.39B 17.41M 4.82 MB 16.00 MB tpcds_1000_parquet.store_sa... Planner Timeline: 4s741ms - Analysis finished: 5.470ms (5.470ms) - Equivalence classes computed: 5.620ms (149.698us) - Single node plan created: 3s881ms (3s875ms) - Runtime filters computed: 3s881ms (32.539us) - Distributed plan created: 3s881ms (396.123us) - Lineage info computed: 3s881ms (550.733us) - Planning finished: 4s741ms (859.114ms) Query Timeline: 36m53s - Start execution: 116.024us (116.024us) - Planning finished: 34m46s (34m46s) - Submit for admission: 34m57s (11s928ms) - Completed admission: 34m57s (70.093us) - Ready to start 7 remote fragment instances: 35m2s (4s046ms) - All 7 remote fragments instances started: 35m38s (36s465ms) - Rows available: 36m52s (1m14s) - First row fetched: 36m52s (336.631ms) - Unregister query: 36m52s (4.636ms) - ComputeScanRangeAssignmentTimer: 7s245ms ImpalaServer: - ClientFetchWaitTimer: 338.114ms - RowMaterializationTimer: 32.542us Execution Profile 3f4226c6295a7e0e:527eeda200000000:(Total: 1m48s, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 8, 25th %-ile: 14s576ms, 50th %-ile: 14s872ms, 75th %-ile: 15s104ms, 90th %-ile: 15s856ms, 95th %-ile: 17s392ms, 99.9th %-ile: 17s392ms Per Node Peak Memory Usage: vd1316.halxg.cloudera.com:22000(4.07 MB) vd1314.halxg.cloudera.com:22000(4.85 MB) vd1311.halxg.cloudera.com:22000(4.69 MB) vd1312.halxg.cloudera.com:22000(4.35 MB) vd1310.halxg.cloudera.com:22000(4.17 MB) vd1313.halxg.cloudera.com:22000(4.11 MB) vd1309.halxg.cloudera.com:22000(4.63 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 1m11s, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(2s000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 30.01 KB, 44.01 KB, 44.01 KB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 48.02 KB (49168) - PerHostPeakMemUsage: 0 - PrepareTime: 33.525ms - RowsProduced: 0 (0) - TotalCpuTime: 39s165ms - TotalNetworkReceiveTime: 1m11s - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 42.990ms, non-child: 0.000ns, % non-child: 0.00%) - CodegenTime: 639.743us - CompileTime: 1.704ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 7 (7) - NumInstructions: 96 (96) - OptimizationTime: 6.506ms - PrepareTime: 34.092ms AGGREGATION_NODE (id=3):(Total: 1m11s, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - BuildTime: 12.987us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.00 KB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 1m11s, non-child: 0.000ns, % non-child: 0.00%) BytesReceived(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 40.00 B, 96.00 B, 96.00 B - BytesReceived: 112.00 B (112) - ConvertRowBatchTime: 50.138us - DeserializeRowBatchTimer: 143.003us - FirstBatchArrivalWaitTime: 1m5s - PeakMemoryUsage: 0 - RowsReturned: 7 (7) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 1m10s, non-child: 4s002ms, % non-child: 5.66%) split sizes: min: 21.99 GB, max: 21.99 GB, avg: 21.99 GB, stddev: 192.98 KB completion times: min:1m13s max:1m16s mean: 1m14s stddev:1s039ms execution rates: min:294.19 MB/sec max:306.74 MB/sec mean:303.63 MB/sec stddev:4.16 MB/sec num instances: 7 - AverageThreadTokens: 73.64 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.41 MB (4622043) - PerHostPeakMemUsage: 4.41 MB (4624969) - PrepareTime: 1s084ms - RowsProduced: 1 (1) - TotalCpuTime: 1h26m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 369.474us - TotalStorageWaitTime: 1h23m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.230ms, non-child: 50.230ms, % non-child: 100.00%) - CodegenTime: 1.000ms - CompileTime: 2.744ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.150ms - PrepareTime: 31.844ms DataStreamSender (dst_id=2):(Total: 94.441us, non-child: 94.441us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 407.88 KB/sec - OverallThroughput: 166.51 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 25.870us - TransmitDataRPCTime: 39.016us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s606ms, % non-child: 5.41%) - BuildTime: 3s279ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m3s, non-child: 1m3s, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.10 - AverageScannerThreadConcurrency: 79.06 - BytesRead: 14.09 GB (15133302407) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15133302407) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15133302407) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.12K (121122) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.37 MB (4579913) - PerReadThreadRawHdfsThroughput: 2.32 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.95M (341952888) - RowsReturned: 341.94M (341940434) - RowsReturnedRate: 5.42 M/sec - ScanRangesComplete: 121.12K (121122) - ScannerThreadsInvoluntaryContextSwitches: 20.20K (20198) - ScannerThreadsTotalWallClockTime: 1h26m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 6s859ms - ScannerThreadsUserTime: 2m53s - ScannerThreadsVoluntaryContextSwitches: 664.31K (664308) - TotalRawHdfsReadTime(*): 6s089ms - TotalReadThroughput: 214.48 MB/sec Fragment F00: Instance 3f4226c6295a7e0e:527eeda200000002 (host=vd1310.halxg.cloudera.com:22000):(Total: 1m12s, non-child: 4s494ms, % non-child: 6.16%) Hdfs split stats (:<# splits>/): 4:15106/2.75 GB 6:14886/2.70 GB 7:15027/2.73 GB 0:15117/2.74 GB 5:15184/2.75 GB 1:15315/2.78 GB 2:15204/2.77 GB 3:15273/2.78 GB MemoryUsage(2s000ms): 0, 0, 7.00 KB, 939.00 KB, 2.20 MB, 2.36 MB, 2.53 MB, 2.50 MB, 2.47 MB, 2.53 MB, 2.49 MB, 2.35 MB, 2.20 MB, 2.36 MB, 2.32 MB, 2.33 MB, 2.42 MB, 2.63 MB, 2.46 MB, 2.29 MB, 2.40 MB, 2.33 MB, 2.40 MB, 2.41 MB, 2.36 MB, 2.56 MB, 2.32 MB, 2.33 MB, 2.42 MB, 2.48 MB, 2.40 MB, 2.27 MB, 2.32 MB, 2.26 MB, 2.33 MB, 2.50 MB ThreadUsage(2s000ms): 1, 1, 1, 41, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.38 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.17 MB (4370432) - PerHostPeakMemUsage: 4.17 MB (4370432) - PrepareTime: 1s114ms - RowsProduced: 1 (1) - TotalCpuTime: 1h28m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 395.939us - TotalStorageWaitTime: 1h25m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 49.841ms, non-child: 49.841ms, % non-child: 100.00%) - CodegenTime: 1.065ms - CompileTime: 2.728ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.092ms - PrepareTime: 31.560ms DataStreamSender (dst_id=2):(Total: 97.437us, non-child: 97.437us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 388.59 KB/sec - OverallThroughput: 160.36 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 27.079us - TransmitDataRPCTime: 40.209us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m8s, non-child: 3s615ms, % non-child: 5.28%) ExecOption: Codegen Enabled - BuildTime: 3s340ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m4s, non-child: 1m4s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121112 out of 121112 Hdfs split stats (:<# splits>/): 4:15106/2.75 GB 6:14886/2.70 GB 7:15027/2.73 GB 0:15117/2.74 GB 5:15184/2.75 GB 1:15315/2.78 GB 2:15204/2.77 GB 3:15273/2.78 GB Hdfs Read Thread Concurrency Bucket: 0:93.28% 1:5.97% 2:0.7463% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121112 BytesRead(2s000ms): 0, 96.11 MB, 564.63 MB, 1020.54 MB, 1.43 GB, 1.90 GB, 2.39 GB, 2.84 GB, 3.30 GB, 3.72 GB, 4.15 GB, 4.52 GB, 4.95 GB, 5.38 GB, 5.81 GB, 6.24 GB, 6.69 GB, 7.11 GB, 7.60 GB, 8.06 GB, 8.47 GB, 8.83 GB, 9.23 GB, 9.65 GB, 10.06 GB, 10.45 GB, 10.83 GB, 11.23 GB, 11.64 GB, 12.03 GB, 12.44 GB, 12.70 GB, 12.98 GB, 13.76 GB - AverageHdfsReadThreadConcurrency: 0.09 - AverageScannerThreadConcurrency: 79.40 - BytesRead: 14.09 GB (15132037643) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15132037643) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15132037643) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.11K (121112) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.12 MB (4317184) - PerReadThreadRawHdfsThroughput: 2.30 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.93M (341926717) - RowsReturned: 341.91M (341914425) - RowsReturnedRate: 5.28 M/sec - ScanRangesComplete: 121.11K (121112) - ScannerThreadsInvoluntaryContextSwitches: 25.08K (25080) - ScannerThreadsTotalWallClockTime: 1h28m - MaterializeTupleTime(*): 1m9s - ScannerThreadsSysTime: 6s311ms - ScannerThreadsUserTime: 2m53s - ScannerThreadsVoluntaryContextSwitches: 659.87K (659867) - TotalRawHdfsReadTime(*): 6s138ms - TotalReadThroughput: 208.91 MB/sec Instance 3f4226c6295a7e0e:527eeda200000007 (host=vd1309.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s762ms, % non-child: 5.33%) Hdfs split stats (:<# splits>/): 7:15021/2.74 GB 0:15137/2.72 GB 2:15258/2.77 GB 1:15137/2.76 GB 3:15075/2.73 GB 4:15134/2.74 GB 6:15168/2.78 GB 5:15157/2.75 GB MemoryUsage(2s000ms): 0, 0, 14.00 KB, 1.88 MB, 2.39 MB, 2.46 MB, 2.31 MB, 2.48 MB, 2.50 MB, 2.44 MB, 2.68 MB, 2.31 MB, 2.47 MB, 2.54 MB, 2.69 MB, 2.48 MB, 2.42 MB, 2.37 MB, 2.38 MB, 2.44 MB, 2.47 MB, 2.51 MB, 2.40 MB, 2.35 MB, 2.28 MB, 2.52 MB, 2.41 MB, 2.41 MB, 2.49 MB, 2.45 MB, 2.45 MB, 2.44 MB, 2.48 MB, 2.46 MB, 2.52 MB ThreadUsage(2s000ms): 1, 1, 1, 61, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.73 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.62 MB (4841472) - PerHostPeakMemUsage: 4.63 MB (4853760) - PrepareTime: 1s130ms - RowsProduced: 1 (1) - TotalCpuTime: 1h26m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 369.674us - TotalStorageWaitTime: 1h23m CodeGen:(Total: 52.204ms, non-child: 52.204ms, % non-child: 100.00%) - CodegenTime: 1.065ms - CompileTime: 2.894ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.555ms - PrepareTime: 33.240ms DataStreamSender (dst_id=2):(Total: 108.274us, non-child: 108.274us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 297.82 KB/sec - OverallThroughput: 144.31 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 27.888us - TransmitDataRPCTime: 52.464us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s632ms, % non-child: 5.44%) ExecOption: Codegen Enabled - BuildTime: 3s337ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m3s, non-child: 1m3s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121087 out of 121087 Hdfs split stats (:<# splits>/): 7:15021/2.74 GB 0:15137/2.72 GB 2:15258/2.77 GB 1:15137/2.76 GB 3:15075/2.73 GB 4:15134/2.74 GB 6:15168/2.78 GB 5:15157/2.75 GB Hdfs Read Thread Concurrency Bucket: 0:90.84% 1:7.634% 2:1.527% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121087 BytesRead(2s000ms): 0, 130.59 MB, 659.70 MB, 1.17 GB, 1.62 GB, 2.04 GB, 2.46 GB, 2.95 GB, 3.42 GB, 3.84 GB, 4.21 GB, 4.64 GB, 5.05 GB, 5.46 GB, 5.89 GB, 6.32 GB, 6.73 GB, 7.17 GB, 7.61 GB, 8.05 GB, 8.54 GB, 8.95 GB, 9.39 GB, 9.80 GB, 10.20 GB, 10.62 GB, 11.01 GB, 11.40 GB, 11.78 GB, 12.20 GB, 12.60 GB, 13.00 GB, 13.42 GB - AverageHdfsReadThreadConcurrency: 0.10 - AverageScannerThreadConcurrency: 79.39 - BytesRead: 14.09 GB (15129981768) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15129981768) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15129981768) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.09K (121087) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.57 MB (4788224) - PerReadThreadRawHdfsThroughput: 2.33 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 342.00M (341996320) - RowsReturned: 341.98M (341983931) - RowsReturnedRate: 5.42 M/sec - ScanRangesComplete: 121.09K (121087) - ScannerThreadsInvoluntaryContextSwitches: 19.03K (19034) - ScannerThreadsTotalWallClockTime: 1h26m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 6s191ms - ScannerThreadsUserTime: 2m55s - ScannerThreadsVoluntaryContextSwitches: 658.93K (658935) - TotalRawHdfsReadTime(*): 6s042ms - TotalReadThroughput: 213.56 MB/sec Instance 3f4226c6295a7e0e:527eeda200000003 (host=vd1312.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s940ms, % non-child: 5.59%) Hdfs split stats (:<# splits>/): 7:15173/2.75 GB 0:15073/2.72 GB 5:15309/2.81 GB 4:15052/2.74 GB 6:15164/2.76 GB 3:15189/2.74 GB 1:14986/2.74 GB 2:15159/2.75 GB MemoryUsage(2s000ms): 0, 0, 14.00 KB, 1.84 MB, 2.42 MB, 2.52 MB, 2.35 MB, 2.42 MB, 2.40 MB, 2.27 MB, 2.46 MB, 2.36 MB, 2.24 MB, 2.46 MB, 2.61 MB, 2.44 MB, 2.36 MB, 2.48 MB, 2.39 MB, 2.44 MB, 2.53 MB, 2.26 MB, 2.48 MB, 2.42 MB, 2.50 MB, 2.40 MB, 2.39 MB, 2.44 MB, 2.35 MB, 2.67 MB, 2.42 MB, 2.35 MB, 2.35 MB, 2.55 MB, 2.46 MB ThreadUsage(2s000ms): 1, 1, 1, 61, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.68 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.34 MB (4554752) - PerHostPeakMemUsage: 4.35 MB (4562944) - PrepareTime: 1s052ms - RowsProduced: 1 (1) - TotalCpuTime: 1h26m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 308.748us - TotalStorageWaitTime: 1h23m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.462ms, non-child: 50.462ms, % non-child: 100.00%) - CodegenTime: 954.075us - CompileTime: 2.750ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.114ms - PrepareTime: 32.042ms DataStreamSender (dst_id=2):(Total: 89.939us, non-child: 89.939us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 424.88 KB/sec - OverallThroughput: 173.73 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 26.410us - TransmitDataRPCTime: 36.775us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s528ms, % non-child: 5.30%) ExecOption: Codegen Enabled - BuildTime: 3s194ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m2s, non-child: 1m2s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121105 out of 121105 Hdfs split stats (:<# splits>/): 7:15173/2.75 GB 0:15073/2.72 GB 5:15309/2.81 GB 4:15052/2.74 GB 6:15164/2.76 GB 3:15189/2.74 GB 1:14986/2.74 GB 2:15159/2.75 GB Hdfs Read Thread Concurrency Bucket: 0:89.23% 1:10.77% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121105 BytesRead(2s000ms): 0, 217.45 MB, 785.72 MB, 1.33 GB, 1.81 GB, 2.24 GB, 2.65 GB, 3.01 GB, 3.43 GB, 3.86 GB, 4.31 GB, 4.74 GB, 5.17 GB, 5.60 GB, 6.05 GB, 6.49 GB, 6.91 GB, 7.36 GB, 7.81 GB, 8.21 GB, 8.61 GB, 9.06 GB, 9.51 GB, 9.94 GB, 10.35 GB, 10.77 GB, 11.17 GB, 11.58 GB, 11.96 GB, 12.38 GB, 12.79 GB, 13.18 GB, 13.60 GB - AverageHdfsReadThreadConcurrency: 0.08 - AverageScannerThreadConcurrency: 79.38 - BytesRead: 14.09 GB (15131030624) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15131030624) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15131030624) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.11K (121105) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.31 MB (4517888) - PerReadThreadRawHdfsThroughput: 2.37 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.88M (341878631) - RowsReturned: 341.87M (341866053) - RowsReturnedRate: 5.43 M/sec - ScanRangesComplete: 121.11K (121105) - ScannerThreadsInvoluntaryContextSwitches: 19.77K (19775) - ScannerThreadsTotalWallClockTime: 1h26m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 6s754ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 666.86K (666860) - TotalRawHdfsReadTime(*): 5s956ms - TotalReadThroughput: 214.73 MB/sec Instance 3f4226c6295a7e0e:527eeda200000005 (host=vd1314.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s971ms, % non-child: 5.65%) Hdfs split stats (:<# splits>/): 6:15290/2.79 GB 3:15186/2.75 GB 0:15049/2.76 GB 5:15131/2.73 GB 2:15099/2.75 GB 4:15232/2.76 GB 1:15122/2.74 GB 7:15002/2.72 GB MemoryUsage(2s000ms): 0, 0, 16.00 KB, 1.88 MB, 2.68 MB, 2.41 MB, 2.29 MB, 2.29 MB, 2.47 MB, 2.50 MB, 2.36 MB, 2.25 MB, 2.43 MB, 2.73 MB, 2.41 MB, 2.41 MB, 2.25 MB, 2.30 MB, 2.47 MB, 2.31 MB, 2.33 MB, 2.48 MB, 2.30 MB, 2.49 MB, 2.60 MB, 2.41 MB, 2.37 MB, 2.39 MB, 2.41 MB, 2.46 MB, 2.40 MB, 2.53 MB, 2.30 MB, 2.39 MB, 2.42 MB ThreadUsage(2s000ms): 1, 1, 1, 61, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.27 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.85 MB (5087232) - PerHostPeakMemUsage: 4.85 MB (5087232) - PrepareTime: 1s074ms - RowsProduced: 1 (1) - TotalCpuTime: 1h25m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 381.696us - TotalStorageWaitTime: 1h22m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.225ms, non-child: 50.225ms, % non-child: 100.00%) - CodegenTime: 997.028us - CompileTime: 2.710ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.114ms - PrepareTime: 31.926ms DataStreamSender (dst_id=2):(Total: 84.205us, non-child: 84.205us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 454.72 KB/sec - OverallThroughput: 185.56 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 21.957us - TransmitDataRPCTime: 34.362us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s673ms, % non-child: 5.54%) ExecOption: Codegen Enabled - BuildTime: 3s317ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m2s, non-child: 1m2s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121111 out of 121111 Hdfs split stats (:<# splits>/): 6:15290/2.79 GB 3:15186/2.75 GB 0:15049/2.76 GB 5:15131/2.73 GB 2:15099/2.75 GB 4:15232/2.76 GB 1:15122/2.74 GB 7:15002/2.72 GB Hdfs Read Thread Concurrency Bucket: 0:90.84% 1:8.397% 2:0.7634% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121111 BytesRead(2s000ms): 0, 224.55 MB, 798.57 MB, 1.33 GB, 1.81 GB, 2.23 GB, 2.65 GB, 3.14 GB, 3.54 GB, 3.97 GB, 4.41 GB, 4.84 GB, 5.26 GB, 5.69 GB, 6.14 GB, 6.58 GB, 6.99 GB, 7.44 GB, 7.88 GB, 8.34 GB, 8.73 GB, 9.16 GB, 9.59 GB, 10.02 GB, 10.42 GB, 10.83 GB, 11.22 GB, 11.62 GB, 12.00 GB, 12.43 GB, 12.84 GB, 13.24 GB, 13.67 GB - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 78.34 - BytesRead: 14.09 GB (15132426605) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15132426605) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15132426605) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.11K (121111) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.82 MB (5050368) - PerReadThreadRawHdfsThroughput: 2.38 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.98M (341983223) - RowsReturned: 341.97M (341970855) - RowsReturnedRate: 5.46 M/sec - ScanRangesComplete: 121.11K (121111) - ScannerThreadsInvoluntaryContextSwitches: 20.49K (20492) - ScannerThreadsTotalWallClockTime: 1h25m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 6s663ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 664.38K (664381) - TotalRawHdfsReadTime(*): 5s925ms - TotalReadThroughput: 215.39 MB/sec Instance 3f4226c6295a7e0e:527eeda200000004 (host=vd1311.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s976ms, % non-child: 5.65%) Hdfs split stats (:<# splits>/): 6:14917/2.70 GB 3:15110/2.73 GB 4:15114/2.72 GB 1:15209/2.78 GB 2:15292/2.80 GB 0:15127/2.76 GB 7:15154/2.74 GB 5:15166/2.76 GB MemoryUsage(2s000ms): 0, 0, 16.00 KB, 1.77 MB, 2.58 MB, 2.44 MB, 2.32 MB, 2.57 MB, 2.46 MB, 2.49 MB, 2.43 MB, 2.43 MB, 2.47 MB, 2.42 MB, 2.44 MB, 2.42 MB, 2.44 MB, 2.32 MB, 2.37 MB, 2.37 MB, 2.41 MB, 2.38 MB, 2.35 MB, 2.51 MB, 2.29 MB, 2.31 MB, 2.31 MB, 2.42 MB, 2.30 MB, 2.29 MB, 2.54 MB, 2.41 MB, 2.36 MB, 2.60 MB, 2.50 MB ThreadUsage(2s000ms): 1, 1, 1, 61, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.62 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.69 MB (4919296) - PerHostPeakMemUsage: 4.69 MB (4919296) - PrepareTime: 1s090ms - RowsProduced: 1 (1) - TotalCpuTime: 1h25m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 386.326us - TotalStorageWaitTime: 1h22m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.006ms, non-child: 50.006ms, % non-child: 100.00%) - CodegenTime: 1.032ms - CompileTime: 2.812ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.067ms - PrepareTime: 31.605ms DataStreamSender (dst_id=2):(Total: 99.030us, non-child: 99.030us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 403.83 KB/sec - OverallThroughput: 157.78 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 26.353us - TransmitDataRPCTime: 38.692us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s586ms, % non-child: 5.41%) ExecOption: Codegen Enabled - BuildTime: 3s253ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m2s, non-child: 1m2s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121089 out of 121089 Hdfs split stats (:<# splits>/): 6:14917/2.70 GB 3:15110/2.73 GB 4:15114/2.72 GB 1:15209/2.78 GB 2:15292/2.80 GB 0:15127/2.76 GB 7:15154/2.74 GB 5:15166/2.76 GB Hdfs Read Thread Concurrency Bucket: 0:87.69% 1:11.54% 2:0.7692% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121089 BytesRead(2s000ms): 0, 187.52 MB, 754.07 MB, 1.23 GB, 1.67 GB, 2.08 GB, 2.52 GB, 3.02 GB, 3.41 GB, 3.84 GB, 4.26 GB, 4.69 GB, 5.11 GB, 5.55 GB, 6.00 GB, 6.42 GB, 6.85 GB, 7.29 GB, 7.73 GB, 8.22 GB, 8.59 GB, 9.02 GB, 9.47 GB, 9.89 GB, 10.29 GB, 10.70 GB, 11.09 GB, 11.47 GB, 11.87 GB, 12.29 GB, 12.69 GB, 13.09 GB, 13.60 GB - AverageHdfsReadThreadConcurrency: 0.16 - AverageScannerThreadConcurrency: 78.77 - BytesRead: 14.09 GB (15131232340) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15131232340) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15131232340) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.09K (121089) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.66 MB (4882432) - PerReadThreadRawHdfsThroughput: 2.28 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 342.16M (342161923) - RowsReturned: 342.15M (342149473) - RowsReturnedRate: 5.46 M/sec - ScanRangesComplete: 121.09K (121089) - ScannerThreadsInvoluntaryContextSwitches: 19.31K (19311) - ScannerThreadsTotalWallClockTime: 1h25m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 6s902ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 663.23K (663234) - TotalRawHdfsReadTime(*): 6s189ms - TotalReadThroughput: 215.84 MB/sec Instance 3f4226c6295a7e0e:527eeda200000001 (host=vd1313.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s919ms, % non-child: 5.58%) Hdfs split stats (:<# splits>/): 7:15168/2.75 GB 1:15040/2.71 GB 2:15123/2.74 GB 4:15153/2.74 GB 3:15249/2.76 GB 6:15064/2.74 GB 0:15239/2.78 GB 5:15176/2.77 GB MemoryUsage(2s000ms): 0, 0, 16.00 KB, 2.57 MB, 2.54 MB, 2.30 MB, 2.39 MB, 2.36 MB, 2.39 MB, 2.50 MB, 2.44 MB, 2.52 MB, 2.51 MB, 2.45 MB, 2.49 MB, 2.38 MB, 2.36 MB, 2.32 MB, 2.38 MB, 2.32 MB, 2.37 MB, 2.25 MB, 2.48 MB, 2.55 MB, 2.37 MB, 2.44 MB, 2.35 MB, 2.28 MB, 2.39 MB, 2.49 MB, 2.37 MB, 2.46 MB, 2.48 MB, 2.52 MB, 2.57 MB ThreadUsage(2s000ms): 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 74.19 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.11 MB (4308992) - PerHostPeakMemUsage: 4.11 MB (4308992) - PrepareTime: 1s078ms - RowsProduced: 1 (1) - TotalCpuTime: 1h25m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 392.669us - TotalStorageWaitTime: 1h22m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 49.497ms, non-child: 49.497ms, % non-child: 100.00%) - CodegenTime: 958.272us - CompileTime: 2.649ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 14.942ms - PrepareTime: 31.445ms DataStreamSender (dst_id=2):(Total: 86.631us, non-child: 86.631us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 450.21 KB/sec - OverallThroughput: 180.36 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 25.643us - TransmitDataRPCTime: 34.706us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s653ms, % non-child: 5.51%) ExecOption: Codegen Enabled - BuildTime: 3s285ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m2s, non-child: 1m2s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121212 out of 121212 Hdfs split stats (:<# splits>/): 7:15168/2.75 GB 1:15040/2.71 GB 2:15123/2.74 GB 4:15153/2.74 GB 3:15249/2.76 GB 6:15064/2.74 GB 0:15239/2.78 GB 5:15176/2.77 GB Hdfs Read Thread Concurrency Bucket: 0:87.69% 1:10.77% 2:1.538% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121212 BytesRead(2s000ms): 0, 259.44 MB, 836.69 MB, 1.36 GB, 1.83 GB, 2.25 GB, 2.67 GB, 3.13 GB, 3.55 GB, 3.98 GB, 4.42 GB, 4.85 GB, 5.27 GB, 5.70 GB, 6.15 GB, 6.59 GB, 6.99 GB, 7.44 GB, 7.88 GB, 8.33 GB, 8.74 GB, 9.18 GB, 9.63 GB, 10.05 GB, 10.45 GB, 10.87 GB, 11.26 GB, 11.66 GB, 12.04 GB, 12.46 GB, 12.87 GB, 13.29 GB, 13.72 GB - AverageHdfsReadThreadConcurrency: 0.10 - AverageScannerThreadConcurrency: 79.38 - BytesRead: 14.10 GB (15141760870) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15141760870) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15141760870) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.21K (121212) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.07 MB (4268032) - PerReadThreadRawHdfsThroughput: 2.18 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.87M (341865918) - RowsReturned: 341.85M (341853323) - RowsReturnedRate: 5.46 M/sec - ScanRangesComplete: 121.21K (121212) - ScannerThreadsInvoluntaryContextSwitches: 19.01K (19006) - ScannerThreadsTotalWallClockTime: 1h25m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 7s280ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 668.32K (668317) - TotalRawHdfsReadTime(*): 6s466ms - TotalReadThroughput: 216.28 MB/sec Instance 3f4226c6295a7e0e:527eeda200000006 (host=vd1316.halxg.cloudera.com:22000):(Total: 1m10s, non-child: 3s951ms, % non-child: 5.64%) Hdfs split stats (:<# splits>/): 0:15316/2.79 GB 7:14945/2.71 GB 4:15111/2.72 GB 6:15133/2.77 GB 3:15243/2.77 GB 1:15186/2.75 GB 5:15174/2.75 GB 2:15033/2.74 GB MemoryUsage(2s000ms): 0, 0, 16.00 KB, 2.03 MB, 2.40 MB, 2.45 MB, 2.54 MB, 2.44 MB, 2.32 MB, 2.74 MB, 2.50 MB, 2.43 MB, 2.27 MB, 2.45 MB, 2.34 MB, 2.26 MB, 2.32 MB, 2.42 MB, 2.36 MB, 2.44 MB, 2.19 MB, 2.45 MB, 2.39 MB, 2.44 MB, 2.34 MB, 2.44 MB, 2.43 MB, 2.37 MB, 2.32 MB, 2.40 MB, 2.23 MB, 2.31 MB, 2.57 MB, 2.54 MB, 2.34 MB ThreadUsage(2s000ms): 1, 1, 1, 61, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 73.62 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.07 MB (4272128) - PerHostPeakMemUsage: 4.07 MB (4272128) - PrepareTime: 1s055ms - RowsProduced: 1 (1) - TotalCpuTime: 1h25m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 351.267us - TotalStorageWaitTime: 1h22m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 49.380ms, non-child: 49.380ms, % non-child: 100.00%) - CodegenTime: 928.216us - CompileTime: 2.668ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.170ms - PrepareTime: 31.088ms DataStreamSender (dst_id=2):(Total: 95.574us, non-child: 95.574us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 435.12 KB/sec - OverallThroughput: 163.49 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 25.765us - TransmitDataRPCTime: 35.910us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 1m6s, non-child: 3s555ms, % non-child: 5.38%) ExecOption: Codegen Enabled - BuildTime: 3s225ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 1m2s, non-child: 1m2s, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121141 out of 121141 Hdfs split stats (:<# splits>/): 0:15316/2.79 GB 7:14945/2.71 GB 4:15111/2.72 GB 6:15133/2.77 GB 3:15243/2.77 GB 1:15186/2.75 GB 5:15174/2.75 GB 2:15033/2.74 GB Hdfs Read Thread Concurrency Bucket: 0:93.08% 1:6.923% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121141 BytesRead(2s000ms): 0, 292.64 MB, 963.23 MB, 1.51 GB, 2.01 GB, 2.46 GB, 2.89 GB, 3.28 GB, 3.68 GB, 4.15 GB, 4.59 GB, 5.01 GB, 5.44 GB, 5.86 GB, 6.30 GB, 6.75 GB, 7.18 GB, 7.60 GB, 8.05 GB, 8.48 GB, 8.92 GB, 9.30 GB, 9.72 GB, 10.17 GB, 10.59 GB, 11.00 GB, 11.42 GB, 11.82 GB, 12.21 GB, 12.61 GB, 13.03 GB, 13.43 GB, 13.84 GB - AverageHdfsReadThreadConcurrency: 0.04 - AverageScannerThreadConcurrency: 78.77 - BytesRead: 14.10 GB (15134647000) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15134647000) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15134647000) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.14K (121141) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.04 MB (4235264) - PerReadThreadRawHdfsThroughput: 2.39 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.86M (341857487) - RowsReturned: 341.84M (341844984) - RowsReturnedRate: 5.47 M/sec - ScanRangesComplete: 121.14K (121141) - ScannerThreadsInvoluntaryContextSwitches: 18.69K (18694) - ScannerThreadsTotalWallClockTime: 1h25m - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 7s911ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 668.57K (668568) - TotalRawHdfsReadTime(*): 5s907ms - TotalReadThroughput: 216.65 MB/sec