impalad /backends /catalog /hadoop-varz /logs /memz /metrics /queries /rpcz /sessions /threadz /varz Query 274b9999d5e3eb74:2a2b021600000000 Plan Query Text plan Summary Profile Memory Download Thrift Profile Query (id=274b9999d5e3eb74:2a2b021600000000): Summary: Session ID: d340fe986ff26978:f2958478fd9c89af Session Type: BEESWAX Start Time: 2016-09-17 11:21:58.351718000 End Time: 2016-09-17 11:52:54.011478000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.8.0-cdh5-INTERNAL DEBUG (build 471d68b76d0c603f2ac0a5184e93395137e7a068) User: mmokhtar Connected User: mmokhtar Delegated User: Network Address: 10.17.234.19:35515 Default Db: tpcds_1000_parquet Sql Statement: select count(*) from store_sales_100kp where ss_item_sk > 10 Coordinator: vd1309.halxg.cloudera.com:22000 Query Options (non default): Plan: ---------------- Estimated Per-Host Requirements: Memory=26.00MB VCores=1 WARNING: The following tables are missing relevant table and/or column statistics. tpcds_1000_parquet.store_sales_100kp 03:AGGREGATE [FINALIZE] | output: count:merge(*) | hosts=7 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 02:EXCHANGE [UNPARTITIONED] | hosts=7 per-host-mem=unavailable | tuple-ids=1 row-size=8B cardinality=1 | 01:AGGREGATE | output: count(*) | hosts=7 per-host-mem=10.00MB | tuple-ids=1 row-size=8B cardinality=1 | 00:SCAN HDFS [tpcds_1000_parquet.store_sales_100kp, RANDOM] partitions=747457/747457 files=847857 size=153.96GB predicates: ss_item_sk > 10 table stats: unavailable column stats: unavailable hosts=7 per-host-mem=16.00MB tuple-ids=0 row-size=8B cardinality=17409513 ---------------- Estimated Per-Host Mem: 27262976 Estimated Per-Host VCores: 1 Tables Missing Stats: tpcds_1000_parquet.store_sales_100kp Request Pool: root.mmokhtar Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------- 03:AGGREGATE 0 0.000ns 0.000ns 0 1 0 -1.00 B FINALIZE 02:EXCHANGE 0 0.000ns 0.000ns 0 1 0 -1.00 B UNPARTITIONED 01:AGGREGATE 7 3s810ms 4s105ms 7 1 44.00 KB 10.00 MB 00:SCAN HDFS 7 43s882ms 49s805ms 2.39B 17.41M 4.89 MB 16.00 MB tpcds_1000_parquet.store_sa... Planner Timeline: 5s849ms - Analysis finished: 47.429ms (47.429ms) - Equivalence classes computed: 67.891ms (20.461ms) - Single node plan created: 1s633ms (1s565ms) - Runtime filters computed: 1s636ms (2.350ms) - Distributed plan created: 1s639ms (3.693ms) - Lineage info computed: 1s654ms (14.256ms) - Planning finished: 5s849ms (4s195ms) Query Timeline: 30m55s - Start execution: 118.666us (118.666us) - Planning finished: 29m8s (29m8s) - Submit for admission: 29m19s (11s036ms) - Completed admission: 29m19s (200.126us) - Ready to start 7 remote fragment instances: 29m22s (3s136ms) - All 7 remote fragments instances started: 29m58s (36s360ms) - Rows available: 30m55s (56s924ms) - First row fetched: 30m55s (153.748ms) - Unregister query: 30m55s (3.170ms) - ComputeScanRangeAssignmentTimer: 7s991ms ImpalaServer: - ClientFetchWaitTimer: 154.856ms - RowMaterializationTimer: 31.671us Execution Profile 274b9999d5e3eb74:2a2b021600000000:(Total: 1m31s, non-child: 0.000ns, % non-child: 0.00%) Number of filters: 0 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed Enabled ---------------------------------------------------------------------------------------------------------------------------- Fragment instance start latencies: Count: 8, 25th %-ile: 13s816ms, 50th %-ile: 14s080ms, 75th %-ile: 14s240ms, 90th %-ile: 14s736ms, 95th %-ile: 16s152ms, 99.9th %-ile: 16s152ms Per Node Peak Memory Usage: vd1310.halxg.cloudera.com:22000(4.79 MB) vd1316.halxg.cloudera.com:22000(4.78 MB) vd1314.halxg.cloudera.com:22000(4.34 MB) vd1312.halxg.cloudera.com:22000(4.47 MB) vd1311.halxg.cloudera.com:22000(4.94 MB) vd1313.halxg.cloudera.com:22000(4.28 MB) vd1309.halxg.cloudera.com:22000(4.97 MB) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns Coordinator Fragment F01:(Total: 54s995ms, non-child: 0.000ns, % non-child: 0.00%) MemoryUsage(2s000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 15.00 KB, 24.01 KB, 27.01 KB, 28.01 KB, 28.01 KB, 28.01 KB, 28.01 KB, 28.01 KB, 36.01 KB, 38.01 KB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 - AverageThreadTokens: 1.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 48.02 KB (49168) - PerHostPeakMemUsage: 0 - PrepareTime: 30.645ms - RowsProduced: 0 (0) - TotalCpuTime: 38s505ms - TotalNetworkReceiveTime: 54s964ms - TotalNetworkSendTime: 0.000ns - TotalStorageWaitTime: 0.000ns BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 48.748ms, non-child: 0.000ns, % non-child: 0.00%) - CodegenTime: 641.922us - CompileTime: 3.106ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 7 (7) - NumInstructions: 96 (96) - OptimizationTime: 13.366ms - PrepareTime: 31.836ms AGGREGATION_NODE (id=3):(Total: 54s995ms, non-child: 0.000ns, % non-child: 0.00%) ExecOption: Codegen Enabled - BuildTime: 10.924us - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 40.00 KB (40960) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) EXCHANGE_NODE (id=2):(Total: 54s964ms, non-child: 0.000ns, % non-child: 0.00%) BytesReceived(2s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4.00 B, 16.00 B, 28.00 B, 32.00 B, 32.00 B, 32.00 B, 32.00 B, 32.00 B, 64.00 B, 72.00 B - BytesReceived: 112.00 B (112) - ConvertRowBatchTime: 75.037us - DeserializeRowBatchTimer: 177.990us - FirstBatchArrivalWaitTime: 35s322ms - PeakMemoryUsage: 0 - RowsReturned: 7 (7) - RowsReturnedRate: 0 - SendersBlockedTimer: 0.000ns - SendersBlockedTotalTimer(*): 0.000ns Averaged Fragment F00:(Total: 51s372ms, non-child: 3s627ms, % non-child: 7.06%) split sizes: min: 21.99 GB, max: 21.99 GB, avg: 21.99 GB, stddev: 162.12 KB completion times: min:42s832ms max:1m mean: 54s674ms stddev:7s166ms execution rates: min:374.44 MB/sec max:525.81 MB/sec mean:420.14 MB/sec stddev:62.75 MB/sec num instances: 7 - AverageThreadTokens: 69.93 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.65 MB (4873069) - PerHostPeakMemUsage: 4.65 MB (4877166) - PrepareTime: 1s080ms - RowsProduced: 1 (1) - TotalCpuTime: 59m40s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 428.363us - TotalStorageWaitTime: 56m22s BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 52.289ms, non-child: 52.289ms, % non-child: 100.00%) - CodegenTime: 977.063us - CompileTime: 3.541ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 16.619ms - PrepareTime: 31.637ms DataStreamSender (dst_id=2):(Total: 136.126us, non-child: 136.126us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 338.58 KB/sec - OverallThroughput: 115.25 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 52.555us - TransmitDataRPCTime: 46.643us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 47s692ms, non-child: 3s810ms, % non-child: 7.99%) - BuildTime: 3s397ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 43s882ms, non-child: 43s882ms, % non-child: 100.00%) - AverageHdfsReadThreadConcurrency: 0.13 - AverageScannerThreadConcurrency: 77.32 - BytesRead: 14.09 GB (15133302407) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15133302407) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15133302407) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.12K (121122) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.60 MB (4823332) - PerReadThreadRawHdfsThroughput: 2.19 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.95M (341952888) - RowsReturned: 341.94M (341940434) - RowsReturnedRate: 8.03 M/sec - ScanRangesComplete: 121.12K (121122) - ScannerThreadsInvoluntaryContextSwitches: 21.78K (21784) - ScannerThreadsTotalWallClockTime: 59m40s - MaterializeTupleTime(*): 1m12s - ScannerThreadsSysTime: 5s896ms - ScannerThreadsUserTime: 2m52s - ScannerThreadsVoluntaryContextSwitches: 724.79K (724792) - TotalRawHdfsReadTime(*): 6s456ms - TotalReadThroughput: 306.54 MB/sec Fragment F00: Instance 274b9999d5e3eb74:2a2b021600000005 (host=vd1314.halxg.cloudera.com:22000):(Total: 57s210ms, non-child: 3s520ms, % non-child: 6.15%) Hdfs split stats (:<# splits>/): 1:15147/2.74 GB 7:14981/2.75 GB 5:15100/2.73 GB 6:15264/2.77 GB 2:15166/2.75 GB 3:15257/2.75 GB 4:15181/2.76 GB 0:15056/2.75 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 14.00 KB, 32.00 KB, 2.12 MB, 2.09 MB, 1.89 MB, 2.09 MB, 2.01 MB, 2.05 MB, 1.90 MB, 2.12 MB, 2.02 MB, 1.96 MB, 2.06 MB, 2.02 MB, 2.09 MB, 2.14 MB, 2.03 MB, 2.10 MB, 2.12 MB, 2.28 MB, 1.97 MB, 2.04 MB, 1.96 MB, 2.19 MB, 1.96 MB, 2.12 MB, 2.29 MB, 2.41 MB, 2.40 MB, 2.42 MB, 2.59 MB, 2.56 MB, 2.35 MB, 2.52 MB, 2.54 MB, 2.59 MB, 2.76 MB, 2.53 MB, 2.61 MB, 2.87 MB, 2.55 MB, 2.53 MB, 2.44 MB, 2.27 MB, 2.39 MB, 2.47 MB, 2.34 MB, 2.41 MB, 2.43 MB, 2.36 MB, 2.33 MB, 1.77 MB, 1.66 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 72.65 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.34 MB (4546560) - PerHostPeakMemUsage: 4.34 MB (4546560) - PrepareTime: 1s055ms - RowsProduced: 1 (1) - TotalCpuTime: 1h9m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 400.433us - TotalStorageWaitTime: 1h5m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 51.921ms, non-child: 51.921ms, % non-child: 100.00%) - CodegenTime: 969.531us - CompileTime: 3.424ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 16.839ms - PrepareTime: 31.216ms DataStreamSender (dst_id=2):(Total: 134.612us, non-child: 134.612us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 302.29 KB/sec - OverallThroughput: 116.07 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 52.290us - TransmitDataRPCTime: 51.689us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 53s637ms, non-child: 3s832ms, % non-child: 7.15%) ExecOption: Codegen Enabled - BuildTime: 3s383ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 49s805ms, non-child: 49s805ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121152 out of 121152 Hdfs split stats (:<# splits>/): 1:15147/2.74 GB 7:14981/2.75 GB 5:15100/2.73 GB 6:15264/2.77 GB 2:15166/2.75 GB 3:15257/2.75 GB 4:15181/2.76 GB 0:15056/2.75 GB Hdfs Read Thread Concurrency Bucket: 0:89.42% 1:9.615% 2:0% 3:0.9615% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121152 BytesRead(1s000ms): 0, 0, 18.21 MB, 133.12 MB, 253.63 MB, 426.59 MB, 645.79 MB, 854.07 MB, 1.03 GB, 1.14 GB, 1.34 GB, 1.54 GB, 1.75 GB, 1.95 GB, 2.16 GB, 2.34 GB, 2.54 GB, 2.72 GB, 2.93 GB, 3.13 GB, 3.31 GB, 3.50 GB, 3.71 GB, 3.92 GB, 4.12 GB, 4.32 GB, 4.52 GB, 4.71 GB, 5.06 GB, 5.44 GB, 5.83 GB, 6.23 GB, 6.63 GB, 6.98 GB, 7.33 GB, 7.73 GB, 8.14 GB, 8.55 GB, 8.95 GB, 9.33 GB, 9.69 GB, 10.06 GB, 10.46 GB, 10.85 GB, 11.25 GB, 11.65 GB, 12.05 GB, 12.42 GB, 12.79 GB, 13.19 GB, 13.59 GB, 13.91 GB, 13.99 GB, 14.05 GB - AverageHdfsReadThreadConcurrency: 0.08 - AverageScannerThreadConcurrency: 79.23 - BytesRead: 14.10 GB (15135848835) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15135848835) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15135848835) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.15K (121152) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.29 MB (4493312) - PerReadThreadRawHdfsThroughput: 2.21 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.88M (341875552) - RowsReturned: 341.86M (341863097) - RowsReturnedRate: 6.86 M/sec - ScanRangesComplete: 121.15K (121152) - ScannerThreadsInvoluntaryContextSwitches: 23.13K (23126) - ScannerThreadsTotalWallClockTime: 1h9m - MaterializeTupleTime(*): 1m12s - ScannerThreadsSysTime: 5s810ms - ScannerThreadsUserTime: 2m53s - ScannerThreadsVoluntaryContextSwitches: 708.40K (708397) - TotalRawHdfsReadTime(*): 6s372ms - TotalReadThroughput: 266.81 MB/sec Instance 274b9999d5e3eb74:2a2b021600000002 (host=vd1313.halxg.cloudera.com:22000):(Total: 57s001ms, non-child: 3s497ms, % non-child: 6.14%) Hdfs split stats (:<# splits>/): 1:15236/2.77 GB 5:15051/2.73 GB 3:15166/2.75 GB 0:15063/2.73 GB 2:15117/2.76 GB 7:15126/2.78 GB 6:15139/2.73 GB 4:15258/2.75 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 14.00 KB, 36.00 KB, 2.16 MB, 2.30 MB, 2.13 MB, 2.31 MB, 2.03 MB, 1.83 MB, 1.97 MB, 1.93 MB, 1.95 MB, 2.09 MB, 2.01 MB, 2.00 MB, 1.98 MB, 2.13 MB, 2.05 MB, 2.09 MB, 2.07 MB, 2.22 MB, 1.97 MB, 2.02 MB, 2.12 MB, 2.07 MB, 1.97 MB, 1.95 MB, 1.96 MB, 2.27 MB, 2.29 MB, 2.52 MB, 2.74 MB, 2.40 MB, 2.67 MB, 2.58 MB, 2.64 MB, 2.31 MB, 2.47 MB, 2.42 MB, 2.17 MB, 2.47 MB, 2.57 MB, 2.60 MB, 2.55 MB, 2.27 MB, 2.53 MB, 2.52 MB, 2.47 MB, 2.29 MB, 2.30 MB, 2.43 MB, 2.07 MB, 1.73 MB, 1.72 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 72.63 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.28 MB (4485120) - PerHostPeakMemUsage: 4.28 MB (4485120) - PrepareTime: 1s030ms - RowsProduced: 1 (1) - TotalCpuTime: 1h8m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 460.698us - TotalStorageWaitTime: 1h5m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 58.621ms, non-child: 58.621ms, % non-child: 100.00%) - CodegenTime: 873.748us - CompileTime: 3.870ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 20.616ms - PrepareTime: 33.609ms DataStreamSender (dst_id=2):(Total: 122.022us, non-child: 122.022us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 347.31 KB/sec - OverallThroughput: 128.05 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 50.246us - TransmitDataRPCTime: 44.988us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 53s444ms, non-child: 3s991ms, % non-child: 7.47%) ExecOption: Codegen Enabled - BuildTime: 3s494ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 49s452ms, non-child: 49s452ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121156 out of 121156 Hdfs split stats (:<# splits>/): 1:15236/2.77 GB 5:15051/2.73 GB 3:15166/2.75 GB 0:15063/2.73 GB 2:15117/2.76 GB 7:15126/2.78 GB 6:15139/2.73 GB 4:15258/2.75 GB Hdfs Read Thread Concurrency Bucket: 0:87.62% 1:11.43% 2:0.9524% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121156 BytesRead(1s000ms): 0, 0, 8.84 MB, 125.35 MB, 307.13 MB, 519.31 MB, 736.99 MB, 941.74 MB, 1.09 GB, 1.30 GB, 1.52 GB, 1.72 GB, 1.93 GB, 2.13 GB, 2.32 GB, 2.53 GB, 2.74 GB, 2.93 GB, 3.13 GB, 3.30 GB, 3.51 GB, 3.71 GB, 3.93 GB, 4.13 GB, 4.34 GB, 4.53 GB, 4.73 GB, 4.93 GB, 5.22 GB, 5.60 GB, 6.00 GB, 6.39 GB, 6.71 GB, 7.10 GB, 7.51 GB, 7.92 GB, 8.33 GB, 8.74 GB, 9.11 GB, 9.44 GB, 9.84 GB, 10.21 GB, 10.61 GB, 11.01 GB, 11.41 GB, 11.79 GB, 12.16 GB, 12.55 GB, 12.95 GB, 13.34 GB, 13.75 GB, 13.95 GB, 14.01 GB, 14.08 GB - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 78.45 - BytesRead: 14.10 GB (15135875885) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15135875885) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15135875885) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.16K (121156) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.23 MB (4431872) - PerReadThreadRawHdfsThroughput: 2.13 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.88M (341878042) - RowsReturned: 341.87M (341865544) - RowsReturnedRate: 6.91 M/sec - ScanRangesComplete: 121.16K (121156) - ScannerThreadsInvoluntaryContextSwitches: 23.60K (23599) - ScannerThreadsTotalWallClockTime: 1h8m - MaterializeTupleTime(*): 1m12s - ScannerThreadsSysTime: 5s918ms - ScannerThreadsUserTime: 2m54s - ScannerThreadsVoluntaryContextSwitches: 707.10K (707096) - TotalRawHdfsReadTime(*): 6s603ms - TotalReadThroughput: 267.21 MB/sec Instance 274b9999d5e3eb74:2a2b021600000004 (host=vd1312.halxg.cloudera.com:22000):(Total: 55s821ms, non-child: 3s538ms, % non-child: 6.34%) Hdfs split stats (:<# splits>/): 3:15194/2.78 GB 5:15097/2.74 GB 2:15186/2.73 GB 1:15149/2.74 GB 6:15037/2.74 GB 4:15185/2.76 GB 0:15127/2.73 GB 7:15124/2.77 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 0, 32.00 KB, 2.27 MB, 2.30 MB, 2.62 MB, 2.46 MB, 2.31 MB, 2.08 MB, 2.04 MB, 2.05 MB, 2.01 MB, 1.90 MB, 2.04 MB, 2.02 MB, 2.00 MB, 2.01 MB, 2.19 MB, 2.01 MB, 1.90 MB, 2.08 MB, 2.12 MB, 2.04 MB, 1.86 MB, 1.99 MB, 1.94 MB, 2.00 MB, 2.19 MB, 2.29 MB, 2.04 MB, 2.04 MB, 2.38 MB, 2.45 MB, 2.37 MB, 2.44 MB, 2.60 MB, 2.71 MB, 2.25 MB, 2.95 MB, 2.42 MB, 2.57 MB, 2.62 MB, 2.70 MB, 2.28 MB, 2.55 MB, 2.39 MB, 2.50 MB, 2.62 MB, 2.46 MB, 2.56 MB, 2.63 MB, 1.63 MB, 1.71 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 72.50 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.47 MB (4689920) - PerHostPeakMemUsage: 4.47 MB (4689920) - PrepareTime: 1s055ms - RowsProduced: 1 (1) - TotalCpuTime: 1h7m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 416.537us - TotalStorageWaitTime: 1h3m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.463ms, non-child: 50.463ms, % non-child: 100.00%) - CodegenTime: 993.642us - CompileTime: 3.557ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.390ms - PrepareTime: 31.052ms DataStreamSender (dst_id=2):(Total: 130.687us, non-child: 130.687us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 351.79 KB/sec - OverallThroughput: 119.56 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 55.000us - TransmitDataRPCTime: 44.416us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 52s231ms, non-child: 3s853ms, % non-child: 7.38%) ExecOption: Codegen Enabled - BuildTime: 3s383ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 48s378ms, non-child: 48s378ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121099 out of 121099 Hdfs split stats (:<# splits>/): 3:15194/2.78 GB 5:15097/2.74 GB 2:15186/2.73 GB 1:15149/2.74 GB 6:15037/2.74 GB 4:15185/2.76 GB 0:15127/2.73 GB 7:15124/2.77 GB Hdfs Read Thread Concurrency Bucket: 0:87.25% 1:11.76% 2:0.9804% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121099 BytesRead(1s000ms): 0, 0, 2.47 MB, 151.73 MB, 368.99 MB, 693.93 MB, 1.08 GB, 1.51 GB, 1.76 GB, 1.93 GB, 2.14 GB, 2.36 GB, 2.57 GB, 2.79 GB, 2.97 GB, 3.18 GB, 3.39 GB, 3.61 GB, 3.82 GB, 4.01 GB, 4.20 GB, 4.39 GB, 4.60 GB, 4.81 GB, 5.02 GB, 5.21 GB, 5.41 GB, 5.62 GB, 5.82 GB, 6.03 GB, 6.24 GB, 6.46 GB, 6.77 GB, 7.17 GB, 7.58 GB, 8.00 GB, 8.41 GB, 8.81 GB, 9.18 GB, 9.56 GB, 9.98 GB, 10.39 GB, 10.79 GB, 11.18 GB, 11.58 GB, 11.92 GB, 12.32 GB, 12.72 GB, 13.13 GB, 13.53 GB, 13.89 GB, 14.00 GB, 14.06 GB - AverageHdfsReadThreadConcurrency: 0.12 - AverageScannerThreadConcurrency: 79.22 - BytesRead: 14.09 GB (15131236241) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15131236241) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15131236241) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.10K (121099) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.44 MB (4653056) - PerReadThreadRawHdfsThroughput: 2.10 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.97M (341972714) - RowsReturned: 341.96M (341960223) - RowsReturnedRate: 7.07 M/sec - ScanRangesComplete: 121.10K (121099) - ScannerThreadsInvoluntaryContextSwitches: 23.13K (23133) - ScannerThreadsTotalWallClockTime: 1h7m - MaterializeTupleTime(*): 1m12s - ScannerThreadsSysTime: 6s196ms - ScannerThreadsUserTime: 2m53s - ScannerThreadsVoluntaryContextSwitches: 708.03K (708034) - TotalRawHdfsReadTime(*): 6s696ms - TotalReadThroughput: 272.02 MB/sec Instance 274b9999d5e3eb74:2a2b021600000003 (host=vd1311.halxg.cloudera.com:22000):(Total: 55s541ms, non-child: 3s615ms, % non-child: 6.51%) Hdfs split stats (:<# splits>/): 3:15157/2.75 GB 7:15103/2.74 GB 6:15135/2.77 GB 2:15014/2.75 GB 1:15035/2.74 GB 4:15246/2.73 GB 0:15116/2.72 GB 5:15283/2.79 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 14.00 KB, 32.00 KB, 2.46 MB, 2.19 MB, 2.54 MB, 2.50 MB, 2.51 MB, 2.04 MB, 2.02 MB, 1.87 MB, 1.99 MB, 1.88 MB, 2.04 MB, 1.99 MB, 2.10 MB, 1.98 MB, 2.10 MB, 2.08 MB, 2.01 MB, 1.76 MB, 2.21 MB, 1.99 MB, 2.18 MB, 1.92 MB, 2.08 MB, 1.80 MB, 2.14 MB, 1.95 MB, 1.91 MB, 2.05 MB, 2.12 MB, 2.62 MB, 2.70 MB, 2.61 MB, 2.72 MB, 2.26 MB, 2.59 MB, 2.42 MB, 2.57 MB, 2.53 MB, 2.62 MB, 2.37 MB, 2.56 MB, 2.62 MB, 2.44 MB, 2.55 MB, 2.32 MB, 2.45 MB, 2.71 MB, 2.24 MB, 1.79 MB, 1.86 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 72.43 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.94 MB (5177344) - PerHostPeakMemUsage: 4.94 MB (5177344) - PrepareTime: 1s082ms - RowsProduced: 1 (1) - TotalCpuTime: 1h6m - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 395.742us - TotalStorageWaitTime: 1h3m BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 51.689ms, non-child: 51.689ms, % non-child: 100.00%) - CodegenTime: 968.535us - CompileTime: 3.585ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 16.073ms - PrepareTime: 31.491ms DataStreamSender (dst_id=2):(Total: 143.584us, non-child: 143.584us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 380.49 KB/sec - OverallThroughput: 108.82 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 70.973us - TransmitDataRPCTime: 41.065us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 51s874ms, non-child: 3s949ms, % non-child: 7.61%) ExecOption: Codegen Enabled - BuildTime: 3s467ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 47s924ms, non-child: 47s924ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121089 out of 121089 Hdfs split stats (:<# splits>/): 3:15157/2.75 GB 7:15103/2.74 GB 6:15135/2.77 GB 2:15014/2.75 GB 1:15035/2.74 GB 4:15246/2.73 GB 0:15116/2.72 GB 5:15283/2.79 GB Hdfs Read Thread Concurrency Bucket: 0:88.12% 1:9.901% 2:1.98% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121089 BytesRead(1s000ms): 0, 0, 10.75 MB, 166.08 MB, 379.58 MB, 721.09 MB, 1.07 GB, 1.48 GB, 1.72 GB, 1.82 GB, 2.04 GB, 2.25 GB, 2.47 GB, 2.69 GB, 2.90 GB, 3.11 GB, 3.33 GB, 3.55 GB, 3.76 GB, 3.96 GB, 4.13 GB, 4.33 GB, 4.55 GB, 4.76 GB, 4.98 GB, 5.19 GB, 5.39 GB, 5.60 GB, 5.81 GB, 6.03 GB, 6.24 GB, 6.50 GB, 6.83 GB, 7.24 GB, 7.65 GB, 8.07 GB, 8.49 GB, 8.91 GB, 9.30 GB, 9.71 GB, 10.13 GB, 10.55 GB, 10.94 GB, 11.34 GB, 11.76 GB, 12.07 GB, 12.48 GB, 12.89 GB, 13.30 GB, 13.71 GB, 13.96 GB, 14.03 GB - AverageHdfsReadThreadConcurrency: 0.12 - AverageScannerThreadConcurrency: 79.21 - BytesRead: 14.09 GB (15130315430) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.09 GB (15130315430) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.09 GB (15130315430) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.09K (121089) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.89 MB (5124096) - PerReadThreadRawHdfsThroughput: 2.22 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 342.00M (341998583) - RowsReturned: 341.99M (341986141) - RowsReturnedRate: 7.14 M/sec - ScanRangesComplete: 121.09K (121089) - ScannerThreadsInvoluntaryContextSwitches: 23.62K (23621) - ScannerThreadsTotalWallClockTime: 1h6m - MaterializeTupleTime(*): 1m12s - ScannerThreadsSysTime: 5s862ms - ScannerThreadsUserTime: 2m53s - ScannerThreadsVoluntaryContextSwitches: 711.99K (711987) - TotalRawHdfsReadTime(*): 6s350ms - TotalReadThroughput: 274.63 MB/sec Instance 274b9999d5e3eb74:2a2b021600000001 (host=vd1309.halxg.cloudera.com:22000):(Total: 53s811ms, non-child: 3s970ms, % non-child: 7.38%) Hdfs split stats (:<# splits>/): 7:15249/2.78 GB 1:14999/2.73 GB 2:15191/2.77 GB 5:14965/2.72 GB 6:15192/2.75 GB 4:15118/2.76 GB 0:15162/2.74 GB 3:15127/2.74 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 0, 28.00 KB, 1.09 MB, 2.22 MB, 1.63 MB, 1.48 MB, 1.56 MB, 1.54 MB, 1.54 MB, 1.91 MB, 1.58 MB, 1.67 MB, 1.52 MB, 1.55 MB, 1.85 MB, 1.70 MB, 1.86 MB, 1.56 MB, 1.56 MB, 1.60 MB, 1.68 MB, 1.71 MB, 1.70 MB, 1.71 MB, 1.67 MB, 1.63 MB, 2.00 MB, 2.10 MB, 2.05 MB, 2.40 MB, 2.57 MB, 2.42 MB, 2.49 MB, 2.56 MB, 2.39 MB, 2.28 MB, 2.35 MB, 3.32 MB, 2.38 MB, 2.40 MB, 2.36 MB, 2.43 MB, 2.56 MB, 2.43 MB, 2.59 MB, 2.48 MB, 2.53 MB, 2.46 MB, 2.58 MB, 2.39 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 30, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 70, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 60.99 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.95 MB (5185536) - PerHostPeakMemUsage: 4.97 MB (5214216) - PrepareTime: 1s099ms - RowsProduced: 1 (1) - TotalCpuTime: 54m10s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 377.716us - TotalStorageWaitTime: 50m40s CodeGen:(Total: 51.520ms, non-child: 51.520ms, % non-child: 100.00%) - CodegenTime: 950.962us - CompileTime: 3.080ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 16.010ms - PrepareTime: 31.919ms DataStreamSender (dst_id=2):(Total: 132.042us, non-child: 132.042us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 302.17 KB/sec - OverallThroughput: 118.33 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 50.296us - TransmitDataRPCTime: 51.710us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 49s789ms, non-child: 4s105ms, % non-child: 8.25%) ExecOption: Codegen Enabled - BuildTime: 3s594ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 45s683ms, non-child: 45s683ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121003 out of 121003 Hdfs split stats (:<# splits>/): 7:15249/2.78 GB 1:14999/2.73 GB 2:15191/2.77 GB 5:14965/2.72 GB 6:15192/2.75 GB 4:15118/2.76 GB 0:15162/2.74 GB 3:15127/2.74 GB Hdfs Read Thread Concurrency Bucket: 0:89.69% 1:8.247% 2:2.062% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121003 BytesRead(1s000ms): 0, 0, 3.73 MB, 162.21 MB, 386.92 MB, 555.61 MB, 750.13 MB, 994.05 MB, 1.22 GB, 1.43 GB, 1.64 GB, 1.88 GB, 2.10 GB, 2.31 GB, 2.49 GB, 2.70 GB, 2.90 GB, 3.10 GB, 3.32 GB, 3.49 GB, 3.71 GB, 3.94 GB, 4.15 GB, 4.35 GB, 4.55 GB, 4.77 GB, 5.07 GB, 5.50 GB, 5.91 GB, 6.22 GB, 6.65 GB, 7.07 GB, 7.46 GB, 7.83 GB, 8.20 GB, 8.50 GB, 8.88 GB, 9.27 GB, 9.66 GB, 10.06 GB, 10.46 GB, 10.76 GB, 11.16 GB, 11.56 GB, 11.97 GB, 12.40 GB, 12.83 GB, 13.15 GB, 13.58 GB, 13.97 GB - AverageHdfsReadThreadConcurrency: 0.11 - AverageScannerThreadConcurrency: 67.41 - BytesRead: 14.08 GB (15122945215) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.08 GB (15122945215) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.08 GB (15122945215) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.00K (121003) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.89 MB (5132288) - PerReadThreadRawHdfsThroughput: 2.04 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 342.22M (342218442) - RowsReturned: 342.21M (342206008) - RowsReturnedRate: 7.49 M/sec - ScanRangesComplete: 121.00K (121003) - ScannerThreadsInvoluntaryContextSwitches: 21.27K (21272) - ScannerThreadsTotalWallClockTime: 54m10s - MaterializeTupleTime(*): 1m14s - ScannerThreadsSysTime: 6s474ms - ScannerThreadsUserTime: 3m3s - ScannerThreadsVoluntaryContextSwitches: 724.39K (724395) - TotalRawHdfsReadTime(*): 6s899ms - TotalReadThroughput: 285.48 MB/sec Instance 274b9999d5e3eb74:2a2b021600000006 (host=vd1316.halxg.cloudera.com:22000):(Total: 40s618ms, non-child: 3s632ms, % non-child: 8.94%) Hdfs split stats (:<# splits>/): 4:15218/2.76 GB 3:15172/2.77 GB 6:15105/2.76 GB 1:15035/2.71 GB 5:15249/2.78 GB 2:14970/2.69 GB 7:15374/2.81 GB 0:15020/2.71 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 14.00 KB, 32.00 KB, 2.44 MB, 2.50 MB, 2.45 MB, 2.52 MB, 2.55 MB, 2.49 MB, 2.27 MB, 2.27 MB, 2.36 MB, 2.65 MB, 2.63 MB, 2.44 MB, 2.59 MB, 2.91 MB, 2.31 MB, 2.62 MB, 2.41 MB, 2.69 MB, 2.45 MB, 2.42 MB, 2.66 MB, 2.60 MB, 2.34 MB, 2.64 MB, 2.81 MB, 2.46 MB, 2.73 MB, 2.54 MB, 2.54 MB, 2.51 MB, 2.54 MB, 2.42 MB, 2.76 MB, 2.64 MB, 2.18 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 69.29 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.78 MB (5009408) - PerHostPeakMemUsage: 4.78 MB (5009408) - PrepareTime: 1s120ms - RowsProduced: 1 (1) - TotalCpuTime: 46m43s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 421.517us - TotalStorageWaitTime: 43m27s BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 50.535ms, non-child: 50.535ms, % non-child: 100.00%) - CodegenTime: 1.111ms - CompileTime: 3.553ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.620ms - PrepareTime: 30.863ms DataStreamSender (dst_id=2):(Total: 139.375us, non-child: 139.375us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 298.97 KB/sec - OverallThroughput: 112.11 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 49.194us - TransmitDataRPCTime: 52.263us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 36s935ms, non-child: 3s461ms, % non-child: 9.37%) ExecOption: Codegen Enabled - BuildTime: 3s173ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 33s473ms, non-child: 33s473ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121143 out of 121143 Hdfs split stats (:<# splits>/): 4:15218/2.76 GB 3:15172/2.77 GB 6:15105/2.76 GB 1:15035/2.71 GB 5:15249/2.78 GB 2:14970/2.69 GB 7:15374/2.81 GB 0:15020/2.71 GB Hdfs Read Thread Concurrency Bucket: 0:78.87% 1:18.31% 2:2.817% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121143 BytesRead(1s000ms): 0, 0, 15.56 MB, 179.40 MB, 433.45 MB, 785.71 MB, 1.17 GB, 1.63 GB, 2.02 GB, 2.39 GB, 2.83 GB, 3.29 GB, 3.75 GB, 4.20 GB, 4.59 GB, 5.04 GB, 5.49 GB, 5.89 GB, 6.35 GB, 6.70 GB, 7.11 GB, 7.57 GB, 8.01 GB, 8.42 GB, 8.85 GB, 9.23 GB, 9.65 GB, 10.06 GB, 10.47 GB, 10.91 GB, 11.34 GB, 11.72 GB, 12.11 GB, 12.53 GB, 12.96 GB, 13.39 GB, 13.82 GB - AverageHdfsReadThreadConcurrency: 0.20 - AverageScannerThreadConcurrency: 78.87 - BytesRead: 14.10 GB (15135167466) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15135167466) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15135167466) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.14K (121143) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.73 MB (4956160) - PerReadThreadRawHdfsThroughput: 2.42 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.88M (341881457) - RowsReturned: 341.87M (341868809) - RowsReturnedRate: 10.21 M/sec - ScanRangesComplete: 121.14K (121143) - ScannerThreadsInvoluntaryContextSwitches: 17.68K (17677) - ScannerThreadsTotalWallClockTime: 46m43s - MaterializeTupleTime(*): 1m11s - ScannerThreadsSysTime: 5s541ms - ScannerThreadsUserTime: 2m47s - ScannerThreadsVoluntaryContextSwitches: 754.25K (754251) - TotalRawHdfsReadTime(*): 5s822ms - TotalReadThroughput: 384.59 MB/sec Instance 274b9999d5e3eb74:2a2b021600000007 (host=vd1310.halxg.cloudera.com:22000):(Total: 39s603ms, non-child: 3s614ms, % non-child: 9.13%) Hdfs split stats (:<# splits>/): 7:15057/2.72 GB 3:15216/2.76 GB 1:15199/2.76 GB 4:15223/2.75 GB 0:15151/2.74 GB 6:15141/2.74 GB 2:15114/2.77 GB 5:15114/2.75 GB MemoryUsage(1s000ms): 0, 0, 0, 0, 2.00 KB, 32.00 KB, 2.56 MB, 2.46 MB, 2.53 MB, 2.37 MB, 2.53 MB, 2.49 MB, 2.45 MB, 2.44 MB, 2.88 MB, 2.65 MB, 2.55 MB, 2.58 MB, 2.48 MB, 3.03 MB, 2.42 MB, 2.59 MB, 2.31 MB, 2.50 MB, 2.56 MB, 2.55 MB, 2.66 MB, 2.29 MB, 2.38 MB, 2.55 MB, 2.55 MB, 2.59 MB, 2.62 MB, 2.54 MB, 2.56 MB, 2.61 MB, 2.49 MB, 2.42 MB, 2.44 MB, 2.33 MB ThreadUsage(1s000ms): 1, 1, 1, 1, 1, 1, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81 - AverageThreadTokens: 69.00 - BloomFilterBytes: 0 - PeakMemoryUsage: 4.79 MB (5017600) - PerHostPeakMemUsage: 4.79 MB (5017600) - PrepareTime: 1s122ms - RowsProduced: 1 (1) - TotalCpuTime: 45m24s - TotalNetworkReceiveTime: 0.000ns - TotalNetworkSendTime: 525.901us - TotalStorageWaitTime: 42m11s BlockMgr: - BlockWritesOutstanding: 0 (0) - BlocksCreated: 0 (0) - BlocksRecycled: 0 (0) - BufferedPins: 0 (0) - BytesWritten: 0 - MaxBlockSize: 8.00 MB (8388608) - MemoryLimit: 160.00 GB (171798691840) - PeakMemoryUsage: 0 - TotalBufferWaitTime: 0.000ns - TotalEncryptionTime: 0.000ns - TotalIntegrityCheckTime: 0.000ns - TotalReadBlockTime: 0.000ns CodeGen:(Total: 51.270ms, non-child: 51.270ms, % non-child: 100.00%) - CodegenTime: 971.630us - CompileTime: 3.717ms - LoadTime: 0.000ns - ModuleBitcodeSize: 1.87 MB (1956236) - NumFunctions: 18 (18) - NumInstructions: 297 (297) - OptimizationTime: 15.783ms - PrepareTime: 31.309ms DataStreamSender (dst_id=2):(Total: 150.560us, non-child: 150.560us, % non-child: 100.00%) - BytesSent: 16.00 B (16) - NetworkThroughput(*): 387.02 KB/sec - OverallThroughput: 103.78 KB/sec - RowsReturned: 1 (1) - SerializeBatchTime: 39.892us - TransmitDataRPCTime: 40.373us - UncompressedRowBatchSize: 16.00 B (16) AGGREGATION_NODE (id=1):(Total: 35s936ms, non-child: 3s477ms, % non-child: 9.68%) ExecOption: Codegen Enabled - BuildTime: 3s287ms - GetResultsTime: 0.000ns - HTResizeTime: 0.000ns - HashBuckets: 0 (0) - LargestPartitionPercent: 0 (0) - MaxPartitionLevel: 0 (0) - NumRepartitions: 0 (0) - PartitionsCreated: 0 (0) - PeakMemoryUsage: 44.00 KB (45056) - RowsRepartitioned: 0 (0) - RowsReturned: 1 (1) - RowsReturnedRate: 0 - SpilledPartitions: 0 (0) HDFS_SCAN_NODE (id=0):(Total: 32s459ms, non-child: 32s459ms, % non-child: 100.00%) ExecOption: Expr Evaluation Codegen Enabled, PARQUET Codegen Enabled, Codegen enabled: 121215 out of 121215 Hdfs split stats (:<# splits>/): 7:15057/2.72 GB 3:15216/2.76 GB 1:15199/2.76 GB 4:15223/2.75 GB 0:15151/2.74 GB 6:15141/2.74 GB 2:15114/2.77 GB 5:15114/2.75 GB Hdfs Read Thread Concurrency Bucket: 0:84.06% 1:15.94% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% File Formats: PARQUET/NONE:121215 BytesRead(1s000ms): 0, 0, 7.06 MB, 146.52 MB, 381.58 MB, 726.04 MB, 1.13 GB, 1.58 GB, 1.97 GB, 2.38 GB, 2.81 GB, 3.29 GB, 3.74 GB, 4.17 GB, 4.59 GB, 5.04 GB, 5.50 GB, 5.96 GB, 6.40 GB, 6.79 GB, 7.26 GB, 7.71 GB, 8.18 GB, 8.64 GB, 9.05 GB, 9.49 GB, 9.92 GB, 10.37 GB, 10.82 GB, 11.24 GB, 11.55 GB, 11.99 GB, 12.45 GB, 12.91 GB, 13.34 GB, 13.77 GB - AverageHdfsReadThreadConcurrency: 0.16 - AverageScannerThreadConcurrency: 78.84 - BytesRead: 14.10 GB (15141727778) - BytesReadDataNodeCache: 0 - BytesReadLocal: 14.10 GB (15141727778) - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 14.10 GB (15141727778) - DecompressionTime: 0.000ns - MaxCompressedTextFileLength: 0 - NumColumns: 1 (1) - NumDisksAccessed: 8 (8) - NumRowGroups: 121.22K (121215) - NumScannerThreadsStarted: 80 (80) - PeakMemoryUsage: 4.74 MB (4972544) - PerReadThreadRawHdfsThroughput: 2.19 GB/sec - RemoteScanRanges: 0 (0) - RowsRead: 341.85M (341845429) - RowsReturned: 341.83M (341833222) - RowsReturnedRate: 10.53 M/sec - ScanRangesComplete: 121.22K (121215) - ScannerThreadsInvoluntaryContextSwitches: 20.06K (20064) - ScannerThreadsTotalWallClockTime: 45m24s - MaterializeTupleTime(*): 1m10s - ScannerThreadsSysTime: 5s476ms - ScannerThreadsUserTime: 2m44s - ScannerThreadsVoluntaryContextSwitches: 759.38K (759384) - TotalRawHdfsReadTime(*): 6s447ms - TotalReadThroughput: 395.02 MB/sec