Query (id=92442b2d9b7951de:381adaf0c8f9a982) Summary Session ID: be4e9af3bb621d4e:9bd2e6a1cf9b8190 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2017-10-09 17:02:29.467901000 End Time: 2017-10-09 17:06:35.237251000 Query Type: QUERY Query State: EXCEPTION Query Status: Cancelled from Impala's debug web interface Impala Version: impalad version 2.6.0-cdh5.8.3 RELEASE (build d016f04ff7d9c22d13fb91a283fff7c1b9e82ed1) User: hive@COE.CLOUDERA.COM Connected User: hive@COE.CLOUDERA.COM Delegated User: Network Address: 10.17.102.128:46680 Default Db: default Sql Statement: SELECT `A`.`a1`, AVG(ALL LENGTH(RTRIM(`A`.`a2`))) as `expr_0` FROM `default`.`test_parquet` `A`, `default`.`test_parquet` `B`, `default`.`test_parquet` WHERE ((`test_parquet`.`a1` = `A`.`a1`) AND (`A`.`a1` = `B`.`a1`)) GROUP BY `A`.`a1` Coordinator: host-10-17-101-152.coe.cloudera.com:22000 Query Options (non default): MEM_LIMIT=900000000,SYNC_DDL=1 Plan: ---------------- Estimated Per-Host Requirements: Memory=4.28GB VCores=3 WARNING: The following tables are missing relevant table and/or column statistics. default.test_parquet 10:EXCHANGE [UNPARTITIONED] | hosts=2 per-host-mem=unavailable | tuple-ids=4 row-size=23B cardinality=unavailable | 09:AGGREGATE [FINALIZE] | output: avg:merge(length(rtrim(A.a2))) | group by: A.a1 | hosts=2 per-host-mem=128.00MB | tuple-ids=4 row-size=23B cardinality=unavailable | 08:EXCHANGE [HASH(A.a1)] | hosts=2 per-host-mem=0B | tuple-ids=3 row-size=23B cardinality=unavailable | 05:AGGREGATE [STREAMING] | output: avg(length(rtrim(A.a2))) | group by: A.a1 | hosts=2 per-host-mem=128.00MB | tuple-ids=3 row-size=23B cardinality=unavailable | 04:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: A.a1 = test_parquet.a1 | runtime filters: RF000 <- test_parquet.a1 | hosts=2 per-host-mem=2.00GB | tuple-ids=0,1,2 row-size=60B cardinality=unavailable | |--07:EXCHANGE [BROADCAST] | | hosts=2 per-host-mem=0B | | tuple-ids=2 row-size=15B cardinality=unavailable | | | 02:SCAN HDFS [default.test_parquet, RANDOM] | partitions=1/1 files=1221 size=7.95GB | table stats: unavailable | column stats: unavailable | hosts=2 per-host-mem=16.00MB | tuple-ids=2 row-size=15B cardinality=unavailable | 03:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: A.a1 = B.a1 | runtime filters: RF001 <- B.a1 | hosts=2 per-host-mem=2.00GB | tuple-ids=0,1 row-size=45B cardinality=unavailable | |--06:EXCHANGE [BROADCAST] | | hosts=2 per-host-mem=0B | | tuple-ids=1 row-size=15B cardinality=unavailable | | | 01:SCAN HDFS [default.test_parquet b, RANDOM] | partitions=1/1 files=1221 size=7.95GB | runtime filters: RF000 -> b.a1 | table stats: unavailable | column stats: unavailable | hosts=2 per-host-mem=16.00MB | tuple-ids=1 row-size=15B cardinality=unavailable | 00:SCAN HDFS [default.test_parquet a, RANDOM] partitions=1/1 files=1221 size=7.95GB runtime filters: RF000 -> A.a1, RF001 -> A.a1 table stats: unavailable column stats: unavailable hosts=2 per-host-mem=32.00MB tuple-ids=0 row-size=30B cardinality=unavailable ---------------- Estimated Per-Host Mem: 4596957184 Estimated Per-Host VCores: 3 Tables Missing Stats: default.test_parquet Request Pool: root.default Admission result: Admitted immediately ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------- 10:EXCHANGE 1 4m5s 4m5s 0 -1 0 -1.00 B UNPARTITIONED 09:AGGREGATE 2 0.000ns 0.000ns 0 -1 90.99 MB 128.00 MB FINALIZE 08:EXCHANGE 2 25.999ms 40.999ms 2.69M -1 0 0 HASH(A.a1) 05:AGGREGATE 2 73.999ms 147.999ms 2.69M -1 6.22 MB 128.00 MB STREAMING 04:HASH JOIN 2 10s724ms 21s448ms 2.72M -1 282.09 MB 2.00 GB INNER JOIN, BROADCAST |--07:EXCHANGE 2 1s161ms 1s188ms 40.00M -1 0 0 BROADCAST | 02:SCAN HDFS 2 22s110ms 41s165ms 40.00M -1 4.96 MB 16.00 MB default.test_parquet 03:HASH JOIN 2 1m15s 2m19s 20.36M -1 282.08 MB 2.00 GB INNER JOIN, BROADCAST |--06:EXCHANGE 2 1s114ms 1s212ms 40.00M -1 0 0 BROADCAST | 01:SCAN HDFS 2 23s724ms 40s798ms 40.00M -1 5.52 MB 16.00 MB default.test_parquet b 00:SCAN HDFS 2 1m53s 3m3s 24.29M -1 8.99 MB 32.00 MB default.test_parquet a Planner Timeline Analysis finished: 9527780 Equivalence classes computed: 10000089 Single node plan created: 15497036 Runtime filters computed: 16231577 Distributed plan created: 19470386 Lineage info computed: 20817497 Planning finished: 25028124 Query Timeline Start execution: 0 Planning finished: 39999948 Submit for admission: 43999944 Completed admission: 43999944 Ready to start 8 remote fragments: 44999944 All 8 remote fragments started: 55999936 First dynamic filter received: 53733957748 Cancelled: 245645803723 Rows available: 245770803622 Unregister query: 245770803622 ImpalaServer - ClientFetchWaitTimer: 0 - InactiveTotalTime: 0 - RowMaterializationTimer: 0 - TotalTime: 0 Execution Profile 92442b2d9b7951de:381adaf0c8f9a982 Number of filters: 2 Filter routing table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed --------------------------------------------------------------------------------------------------------------------- 1 3 0 2 LOCAL false 0 (2) N/A N/A 0 4 0, 1 2, 2 LOCAL, REMOTE false, false 1 (2) N/A N/A Fragment start latencies: Count: 8, 25th %-ile: 1ms, 50th %-ile: 3ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile: 7ms, 99.9th %-ile: 7ms Per Node Peak Memory Usage: host-10-17-101-153.coe.cloudera.com:22000(761.88 MB) host-10-17-101-152.coe.cloudera.com:22000(739.85 MB) Final filter table: ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed --------------------------------------------------------------------------------------------------------------------- 1 3 0 2 LOCAL false 0 (2) N/A N/A 0 4 0, 1 2, 2 LOCAL, REMOTE false, false 0 (2) 53s733ms 53s733ms - FiltersReceived: 1 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 245725803679 Coordinator Fragment F04 - AverageThreadTokens: 0.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - PerHostPeakMemUsage: 0 - PrepareTime: 0 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 245588803788 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 245589803788 BlockMgr - BlockWritesOutstanding: 2 - BlocksCreated: 576 - BlocksRecycled: 43 - BufferedPins: 0 - BytesWritten: 2927624192 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 720000000 - PeakMemoryUsage: 719585280 - TotalBufferWaitTime: 5270995776 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=10) - BytesReceived: 0 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 0 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 245588803788 Averaged Fragment F03 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:4m5s max:4m5s mean: 4m5s stddev:0.000ns execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 2 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 89336048 - PerHostPeakMemUsage: 787337312 - PrepareTime: 149499678 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 238770461256 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 149999677 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 789 - BlocksRecycled: 5045 - BufferedPins: 345 - BytesWritten: 10687086592 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 720000000 - PeakMemoryUsage: 719945728 - TotalBufferWaitTime: 18708930647 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 121524548818 - TotalTime: 0 CodeGen - CodegenTime: 999998 - CompileTime: 31999945 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 44 - NumInstructions: 915 - OptimizationTime: 128999709 - PrepareTime: 147999680 - TotalTime: 308999335 DataStreamSender (dst_id=10) - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 4096 - RowsReturned: 0 - SerializeBatchTime: 0 - TotalTime: 0 - TransmitDataRPCTime: 0 - UncompressedRowBatchSize: 0 AGGREGATION_NODE (id=9) - BuildTime: 306999494 - GetNewBlockTime: 118499902 - GetResultsTime: 0 - HTResizeTime: 8999990 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 89160632 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 3 - TotalTime: 149499678 - UnpinTime: 0 EXCHANGE_NODE (id=8) - BytesReceived: 41724864 - ConvertRowBatchTime: 10999968 - DeserializeRowBatchTimer: 112499739 - FirstBatchArrivalWaitTime: 220938501790 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1343360 - RowsReturnedRate: 5625 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 238780961227 Averaged Fragment F00 split sizes: min: 3.97 GB, max: 3.98 GB, avg: 3.98 GB, stddev: 2.33 MB completion times: min:4m5s max:4m5s mean: 4m5s stddev:1.999ms execution rates: min:16.56 MB/sec max:16.58 MB/sec mean:16.57 MB/sec stddev:9.57 KB/sec num instances: 2 - AverageThreadTokens: 1.8163943369157145 - BloomFilterBytes: 2097152 - InactiveTotalTime: 0 - PeakMemoryUsage: 615288404 - PerHostPeakMemUsage: 787337312 - PrepareTime: 159499640 - RowsProduced: 1343613 - TotalCpuTime: 102956195358 - TotalNetworkReceiveTime: 79530330469 - TotalNetworkSendTime: 227999150 - TotalStorageWaitTime: 110374850667 - TotalTime: 119403056961 CodeGen - CodegenTime: 999997 - CompileTime: 207999536 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 126 - NumInstructions: 6007 - OptimizationTime: 464498982 - PrepareTime: 146999667 - TotalTime: 820498184 DataStreamSender (dst_id=8) - BytesSent: 41724864 - InactiveTotalTime: 0 - NetworkThroughput(*): 121647284 - OverallThroughput: 54973675 - PeakMemoryUsage: 67984 - RowsReturned: 1343613 - SerializeBatchTime: 207999222 - TotalTime: 379498587 - TransmitDataRPCTime: 171499364 - UncompressedRowBatchSize: 96585390 AGGREGATION_NODE (id=5) - GetNewBlockTime: 1999993 - GetResultsTime: 0 - HTResizeTime: 14499937 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 3925120 - PinTime: 0 - ReductionFactorEstimate: 0.5554398826979472 - ReductionFactorThresholdToExpand: 0.55 - RowsPassedThrough: 1343613 - RowsReturned: 1343613 - RowsReturnedRate: 5667 - StreamingTime: 357998673 - TotalTime: 118607559914 - UnpinTime: 0 HASH_JOIN_NODE (id=4) - BuildPartitionTime: 11007472764 - BuildRows: 40000000 - BuildRowsPartitioned: 43752335 - BuildTime: 97999636 - GetNewBlockTime: 4010488137 - HashBuckets: 4325376 - HashCollisions: 1221 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 10724460104 - MaxPartitionLevel: 0 - NumRepartitions: 16 - PartitionsCreated: 40 - PeakMemoryUsage: 295789584 - PinTime: 8736467536 - ProbeRows: 10181067 - ProbeRowsPartitioned: 3104536 - ProbeTime: 1182495584 - RowsReturned: 1362254 - RowsReturnedRate: 5730 - SpilledPartitions: 38 - TotalTime: 118862558749 - UnpinTime: 3999988 EXCHANGE_NODE (id=7) - BytesReceived: 852867267 - ConvertRowBatchTime: 587998671 - DeserializeRowBatchTimer: 1467496731 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 951196 - SendersBlockedTimer: 1879494249 - SendersBlockedTotalTimer(*): 3321489626 - TotalTime: 42099407422 HASH_JOIN_NODE (id=3) - BuildPartitionTime: 15674455396 - BuildRows: 40000000 - BuildRowsPartitioned: 60000000 - BuildTime: 728497302 - GetNewBlockTime: 10926467030 - HashBuckets: 33554432 - HashCollisions: 9718 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 75419736646 - MaxPartitionLevel: 0 - NumRepartitions: 128 - PartitionsCreated: 144 - PeakMemoryUsage: 295779360 - PinTime: 52242806073 - ProbeRows: 12142717 - ProbeRowsPartitioned: 19364920 - ProbeTime: 5885984032 - RowsReturned: 10181067 - RowsReturnedRate: 47069 - SpilledPartitions: 136 - TotalTime: 227285503447 - UnpinTime: 1499995 EXCHANGE_NODE (id=6) - BytesReceived: 852862134 - ConvertRowBatchTime: 597498677 - DeserializeRowBatchTimer: 1348497034 - FirstBatchArrivalWaitTime: 189499542 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 1050518 - SendersBlockedTimer: 3672486379 - SendersBlockedTotalTimer(*): 5357980123 - TotalTime: 38707920190 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.6549253731343283 - AverageScannerThreadConcurrency: 0.736318407960199 - BytesRead: 645719566 - BytesReadDataNodeCache: 0 - BytesReadLocal: 531249723 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 531249723 - DecompressionTime: 12499962 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 0 - NumRowGroups: 371 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 9425652 - PerReadThreadRawHdfsThroughput: 15930203 - RemoteScanRanges: 0 - RowsRead: 12142717 - RowsReturned: 12142717 - RowsReturnedRate: 243586 - ScanRangesComplete: 370 - ScannerThreadsInvoluntaryContextSwitches: 5303 - ScannerThreadsTotalWallClockTime: 23564412565 - MaterializeTupleTime(*): 1407495453 - ScannerThreadsSysTime: 607907500 - ScannerThreadsUserTime: 1174821000 - ScannerThreadsVoluntaryContextSwitches: 1420 - TotalRawHdfsReadTime(*): 106110864310 - TotalReadThroughput: 5722420 - TotalTime: 113157846610 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Filter 1 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Averaged Fragment F02 split sizes: min: 3.97 GB, max: 3.98 GB, avg: 3.98 GB, stddev: 2.33 MB completion times: min:16s341ms max:53s685ms mean: 35s013ms stddev:18s671ms execution rates: min:75.80 MB/sec max:249.31 MB/sec mean:162.55 MB/sec stddev:86.75 MB/sec num instances: 2 - AverageThreadTokens: 2.102803738317757 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 4942440 - PerHostPeakMemUsage: 787337312 - PrepareTime: 1000003 - RowsProduced: 20000000 - TotalCpuTime: 42227371978 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 5947986711 - TotalStorageWaitTime: 31525393572 - TotalTime: 34814394576 DataStreamSender (dst_id=7) - BytesSent: 852867267 - InactiveTotalTime: 0 - NetworkThroughput(*): 746037713 - OverallThroughput: 162440960 - PeakMemoryUsage: 10912 - RowsReturned: 20000000 - SerializeBatchTime: 4099991032 - TotalTime: 5264488567 - TransmitDataRPCTime: 1151997556 - UncompressedRowBatchSize: 1595714339 HDFS_SCAN_NODE (id=2) - AverageHdfsReadThreadConcurrency: 0.04318889832908525 - AverageScannerThreadConcurrency: 1.102803738317757 - BytesRead: 334856151 - BytesReadDataNodeCache: 0 - BytesReadLocal: 334856151 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 334856151 - DecompressionTime: 12499975 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 610 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 4578724 - PerReadThreadRawHdfsThroughput: 1694792548 - RemoteScanRanges: 0 - RowsRead: 20000000 - RowsReturned: 20000000 - RowsReturnedRate: 3516895 - ScanRangesComplete: 610 - ScannerThreadsInvoluntaryContextSwitches: 2613 - ScannerThreadsTotalWallClockTime: 40619373235 - MaterializeTupleTime(*): 6745985373 - ScannerThreadsSysTime: 530418500 - ScannerThreadsUserTime: 1403786000 - ScannerThreadsVoluntaryContextSwitches: 11886 - TotalRawHdfsReadTime(*): 2370491367 - TotalReadThroughput: 13028069 - TotalTime: 22110922467 Averaged Fragment F01 split sizes: min: 3.97 GB, max: 3.98 GB, avg: 3.98 GB, stddev: 2.33 MB completion times: min:24s058ms max:53s803ms mean: 38s931ms stddev:14s872ms execution rates: min:75.64 MB/sec max:169.34 MB/sec mean:122.49 MB/sec stddev:46.85 MB/sec num instances: 2 - AverageThreadTokens: 2.219528816199377 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 5511780 - PerHostPeakMemUsage: 787337312 - PrepareTime: 1499998 - RowsProduced: 20000000 - TotalCpuTime: 47637370494 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 8035984794 - TotalStorageWaitTime: 32539897729 - TotalTime: 38696391411 DataStreamSender (dst_id=6) - BytesSent: 852862134 - InactiveTotalTime: 0 - NetworkThroughput(*): 727596830 - OverallThroughput: 158441671 - PeakMemoryUsage: 10912 - RowsReturned: 20000000 - SerializeBatchTime: 4200990794 - TotalTime: 5396988281 - TransmitDataRPCTime: 1180497511 - UncompressedRowBatchSize: 1595714339 HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.057145638629283485 - AverageScannerThreadConcurrency: 1.219528816199377 - BytesRead: 334856151 - BytesReadDataNodeCache: 0 - BytesReadLocal: 334856151 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 334856151 - DecompressionTime: 13499968 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 610 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 5492676 - PerReadThreadRawHdfsThroughput: 1384388699 - RemoteScanRanges: 0 - RowsRead: 20000000 - RowsReturned: 20000000 - RowsReturnedRate: 1749580 - ScanRangesComplete: 610 - ScannerThreadsInvoluntaryContextSwitches: 4000 - ScannerThreadsTotalWallClockTime: 46882871081 - MaterializeTupleTime(*): 12095978983 - ScannerThreadsSysTime: 519420500 - ScannerThreadsUserTime: 1619753000 - ScannerThreadsVoluntaryContextSwitches: 12218 - TotalRawHdfsReadTime(*): 1590494270 - TotalReadThroughput: 9621027 - TotalTime: 23724421740 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 9920114 - Rows rejected: 0 - Rows total: 9920719 - TotalTime: 0 Fragment F03 Instance 92442b2d9b7951de:381adaf0c8f9a984 (host=host-10-17-101-153.coe.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 95683040 - PerHostPeakMemUsage: 798888872 - PrepareTime: 152999470 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 238909113069 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 152999470 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 789 - BlocksRecycled: 5045 - BufferedPins: 345 - BytesWritten: 10687086592 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 720000000 - PeakMemoryUsage: 719945728 - TotalBufferWaitTime: 18708930647 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 121524548818 - TotalTime: 0 CodeGen - CodegenTime: 999997 - CompileTime: 19999925 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 44 - NumInstructions: 915 - OptimizationTime: 128999520 - PrepareTime: 150999476 - TotalTime: 299998921 DataStreamSender (dst_id=10) - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 4096 - RowsReturned: 0 - SerializeBatchTime: 0 - TotalTime: 0 - TransmitDataRPCTime: 0 - UncompressedRowBatchSize: 0 AGGREGATION_NODE (id=9) ExecOption: Codegen Enabled, Spilled - BuildTime: 177999342 - GetNewBlockTime: 999997 - GetResultsTime: 0 - HTResizeTime: 999996 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 95406576 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 5 - TotalTime: 152999470 - UnpinTime: 0 EXCHANGE_NODE (id=8) - BytesReceived: 41679683 - ConvertRowBatchTime: 13999945 - DeserializeRowBatchTimer: 120999557 - FirstBatchArrivalWaitTime: 220985179703 - InactiveTotalTime: 238883113166 - PeakMemoryUsage: 0 - RowsReturned: 1341520 - RowsReturnedRate: 5614 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 238924113012 Instance 92442b2d9b7951de:381adaf0c8f9a985 (host=host-10-17-101-152.coe.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 82989056 - PerHostPeakMemUsage: 775785752 - PrepareTime: 145999886 - RowsProduced: 0 - TotalCpuTime: 0 - TotalNetworkReceiveTime: 238631809444 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 146999885 CodeGen - CodegenTime: 1000000 - CompileTime: 43999966 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 44 - NumInstructions: 915 - OptimizationTime: 128999898 - PrepareTime: 144999885 - TotalTime: 317999750 DataStreamSender (dst_id=10) - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 4096 - RowsReturned: 0 - SerializeBatchTime: 0 - TotalTime: 0 - TransmitDataRPCTime: 0 - UncompressedRowBatchSize: 0 AGGREGATION_NODE (id=9) ExecOption: Codegen Enabled, Spilled - BuildTime: 435999646 - GetNewBlockTime: 235999807 - GetResultsTime: 0 - HTResizeTime: 16999985 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 82914688 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 2 - TotalTime: 145999886 - UnpinTime: 0 EXCHANGE_NODE (id=8) - BytesReceived: 41770045 - ConvertRowBatchTime: 7999992 - DeserializeRowBatchTimer: 103999921 - FirstBatchArrivalWaitTime: 220891823877 - InactiveTotalTime: 238626809448 - PeakMemoryUsage: 0 - RowsReturned: 1345201 - RowsReturnedRate: 5636 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 238637809442 Fragment F00 Instance 92442b2d9b7951de:381adaf0c8f9a986 (host=host-10-17-101-153.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:610/3.97 GB Filter 0 arrival: 53s769ms Filter 1 arrival: 53s801ms - AverageThreadTokens: 1.6369294605809128 - BloomFilterBytes: 2097152 - InactiveTotalTime: 0 - PeakMemoryUsage: 615893464 - PerHostPeakMemUsage: 798888872 - PrepareTime: 156999409 - RowsProduced: 2687226 - TotalCpuTime: 153083432238 - TotalNetworkReceiveTime: 73182728490 - TotalNetworkSendTime: 455998300 - TotalStorageWaitTime: 41709844874 - TotalTime: 238643114052 CodeGen - CodegenTime: 999997 - CompileTime: 204999239 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 126 - NumInstructions: 6007 - OptimizationTime: 447998339 - PrepareTime: 144999453 - TotalTime: 798997028 DataStreamSender (dst_id=8) - BytesSent: 83449728 - InactiveTotalTime: 0 - NetworkThroughput(*): 243294569 - OverallThroughput: 109947350 - PeakMemoryUsage: 129424 - RowsReturned: 2687226 - SerializeBatchTime: 415998445 - TotalTime: 758997174 - TransmitDataRPCTime: 342998729 - UncompressedRowBatchSize: 193170781 AGGREGATION_NODE (id=5) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 3999986 - GetResultsTime: 0 - HTResizeTime: 28999874 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 6517888 - PinTime: 0 - ReductionFactorEstimate: 1.1108797653958944 - ReductionFactorThresholdToExpand: 1.1 - RowsPassedThrough: 2687226 - RowsReturned: 2687226 - RowsReturnedRate: 11335 - StreamingTime: 715997347 - TotalTime: 237053119958 - UnpinTime: 0 HASH_JOIN_NODE (id=4) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Hash Table Construction Codegen Enabled, Join Build-Side Prepared Asynchronously, Spilled, 0 of 1 Runtime Filter Published, 1 Disabled - BuildPartitionTime: 12736952750 - BuildRows: 40000000 - BuildRowsPartitioned: 47504671 - BuildTime: 195999273 - GetNewBlockTime: 5963977894 - HashBuckets: 8650752 - HashCollisions: 2443 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 21448920208 - MaxPartitionLevel: 1 - NumRepartitions: 33 - PartitionsCreated: 64 - PeakMemoryUsage: 295789600 - PinTime: 17472935072 - ProbeRows: 20362134 - ProbeRowsPartitioned: 6209073 - ProbeTime: 2364991168 - RowsReturned: 2724508 - RowsReturnedRate: 11461 - SpilledPartitions: 61 - TotalTime: 237711117509 - UnpinTime: 3999984 EXCHANGE_NODE (id=7) - BytesReceived: 852867267 - ConvertRowBatchTime: 593997808 - DeserializeRowBatchTimer: 1449994619 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 39556853281 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 983022 - SendersBlockedTimer: 2923989163 - SendersBlockedTotalTimer(*): 5316980300 - TotalTime: 40690849107 HASH_JOIN_NODE (id=3) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Hash Table Construction Codegen Enabled, Join Build-Side Prepared Asynchronously, Spilled, 0 of 1 Runtime Filter Published, 1 Disabled - BuildPartitionTime: 22119918071 - BuildRows: 40000000 - BuildRowsPartitioned: 80000000 - BuildTime: 1456994604 - GetNewBlockTime: 16656938224 - HashBuckets: 67108864 - HashCollisions: 19437 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 139431482356 - MaxPartitionLevel: 1 - NumRepartitions: 256 - PartitionsCreated: 272 - PeakMemoryUsage: 295777328 - PinTime: 104485612146 - ProbeRows: 19988634 - ProbeRowsPartitioned: 38729840 - ProbeTime: 7639971445 - RowsReturned: 20362134 - RowsReturnedRate: 94138 - SpilledPartitions: 256 - TotalTime: 216300197165 - UnpinTime: 2999990 EXCHANGE_NODE (id=6) - BytesReceived: 852862134 - ConvertRowBatchTime: 588997812 - DeserializeRowBatchTimer: 1342995096 - FirstBatchArrivalWaitTime: 210999216 - InactiveTotalTime: 32746878427 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 1184698 - SendersBlockedTimer: 7344972758 - SendersBlockedTotalTimer(*): 10715960246 - TotalTime: 33763874673 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 610 Hdfs split stats (:<# splits>/): 0:610/3.97 GB Runtime filters: All filters arrived. Waited 0 Hdfs Read Thread Concurrency Bucket: 0:63.68% 1:36.32% 2:0% 3:0% File Formats: PARQUET/SNAPPY:1220 - AverageHdfsReadThreadConcurrency: 0.36318407960199006 - AverageScannerThreadConcurrency: 0.472636815920398 - BytesRead: 1062499447 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1062499447 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1062499447 - DecompressionTime: 21999925 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 610 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 9425808 - PerReadThreadRawHdfsThroughput: 30570358 - RemoteScanRanges: 0 - RowsRead: 19988634 - RowsReturned: 19988634 - RowsReturnedRate: 463721 - ScanRangesComplete: 610 - ScannerThreadsInvoluntaryContextSwitches: 10606 - ScannerThreadsTotalWallClockTime: 47128825131 - MaterializeTupleTime(*): 2336991334 - ScannerThreadsSysTime: 1215815000 - ScannerThreadsUserTime: 2349642000 - ScannerThreadsVoluntaryContextSwitches: 2841 - TotalRawHdfsReadTime(*): 34755870885 - TotalReadThroughput: 10502123 - TotalTime: 43104840136 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Filter 1 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Instance 92442b2d9b7951de:381adaf0c8f9a987 (host=host-10-17-101-152.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:611/3.98 GB Filter 0 arrival: 53s684ms Filter 1 arrival: 53s804ms - AverageThreadTokens: 1.9958592132505175 - BloomFilterBytes: 2097152 - InactiveTotalTime: 0 - PeakMemoryUsage: 614683344 - PerHostPeakMemUsage: 775785752 - PrepareTime: 161999871 - RowsProduced: 0 - TotalCpuTime: 52828958478 - TotalNetworkReceiveTime: 85877932449 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 179039856460 - TotalTime: 162999871 CodeGen - CodegenTime: 999998 - CompileTime: 210999833 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 2254648 - NumFunctions: 126 - NumInstructions: 6007 - OptimizationTime: 480999626 - PrepareTime: 148999882 - TotalTime: 841999341 DataStreamSender (dst_id=8) - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 6544 - RowsReturned: 0 - SerializeBatchTime: 0 - TotalTime: 0 - TransmitDataRPCTime: 0 - UncompressedRowBatchSize: 0 AGGREGATION_NODE (id=5) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 0 - GetResultsTime: 0 - HTResizeTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 1332352 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - StreamingTime: 0 - TotalTime: 161999871 - UnpinTime: 0 HASH_JOIN_NODE (id=4) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Hash Table Construction Codegen Enabled, Join Build-Side Prepared Asynchronously, Spilled, 0 of 1 Runtime Filter Published, 1 Disabled - BuildPartitionTime: 9277992778 - BuildRows: 40000000 - BuildRowsPartitioned: 40000000 - BuildTime: 0 - GetNewBlockTime: 2056998380 - HashBuckets: 0 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 295789568 - PinTime: 0 - ProbeRows: 0 - ProbeRowsPartitioned: 0 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 16 - TotalTime: 13999990 - UnpinTime: 3999992 EXCHANGE_NODE (id=7) - BytesReceived: 852867267 - ConvertRowBatchTime: 581999534 - DeserializeRowBatchTimer: 1484998844 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 42318966693 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 919371 - SendersBlockedTimer: 834999336 - SendersBlockedTotalTimer(*): 1325998952 - TotalTime: 43507965738 HASH_JOIN_NODE (id=3) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled, 0 of 1 Runtime Filter Published, 1 Disabled - BuildPartitionTime: 9228992721 - BuildRows: 40000000 - BuildRowsPartitioned: 40000000 - BuildTime: 0 - GetNewBlockTime: 5195995836 - HashBuckets: 0 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 11407990937 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 295781392 - PinTime: 0 - ProbeRows: 4296801 - ProbeRowsPartitioned: 0 - ProbeTime: 4131996619 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 16 - TotalTime: 238270809729 - UnpinTime: 0 EXCHANGE_NODE (id=6) - BytesReceived: 852862134 - ConvertRowBatchTime: 605999542 - DeserializeRowBatchTimer: 1353998973 - FirstBatchArrivalWaitTime: 167999868 - InactiveTotalTime: 42438966644 - PeakMemoryUsage: 0 - RowsReturned: 40000000 - RowsReturnedRate: 916339 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 43651965707 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled Hdfs split stats (:<# splits>/): 0:611/3.98 GB Runtime filters: All filters arrived. Waited 0 - AverageHdfsReadThreadConcurrency: 0.9466666666666667 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 228939685 - BytesReadDataNodeCache: 0 - BytesReadLocal: 0 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 0 - DecompressionTime: 2999999 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 0 - NumRowGroups: 132 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 9425496 - PerReadThreadRawHdfsThroughput: 1290049 - RemoteScanRanges: 0 - RowsRead: 4296801 - RowsReturned: 4296801 - RowsReturnedRate: 23452 - ScanRangesComplete: 131 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 0 - MaterializeTupleTime(*): 477999573 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 0 - TotalRawHdfsReadTime(*): 177465857736 - TotalReadThroughput: 942717 - TotalTime: 183210853085 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Filter 1 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 0 - Rows rejected: 0 - Rows total: 0 - TotalTime: 0 Fragment F02 Instance 92442b2d9b7951de:381adaf0c8f9a988 (host=host-10-17-101-153.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:610/3.97 GB - AverageThreadTokens: 2.205607476635514 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 5799064 - PerHostPeakMemUsage: 798888872 - PrepareTime: 1000008 - RowsProduced: 19988634 - TotalCpuTime: 64932759245 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 5913978073 - TotalStorageWaitTime: 55909792780 - TotalTime: 53457801813 DataStreamSender (dst_id=7) - BytesSent: 852410566 - InactiveTotalTime: 0 - NetworkThroughput(*): 811050188 - OverallThroughput: 170824393 - PeakMemoryUsage: 10912 - RowsReturned: 19988634 - SerializeBatchTime: 3929985474 - TotalTime: 4989981518 - TransmitDataRPCTime: 1050996077 - UncompressedRowBatchSize: 1594805262 HDFS_SCAN_NODE (id=2) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 610 Hdfs split stats (:<# splits>/): 0:610/3.97 GB Hdfs Read Thread Concurrency Bucket: 0:94.39% 1:5.607% 2:0% 3:0% File Formats: PARQUET/SNAPPY:610 - AverageHdfsReadThreadConcurrency: 0.056074766355140186 - AverageScannerThreadConcurrency: 1.205607476635514 - BytesRead: 334648896 - BytesReadDataNodeCache: 0 - BytesReadLocal: 334648896 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 334648896 - DecompressionTime: 8999964 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 610 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 5197232 - PerReadThreadRawHdfsThroughput: 72122873 - RemoteScanRanges: 0 - RowsRead: 19988634 - RowsReturned: 19988634 - RowsReturnedRate: 485563 - ScanRangesComplete: 610 - ScannerThreadsInvoluntaryContextSwitches: 2486 - ScannerThreadsTotalWallClockTime: 64932759245 - MaterializeTupleTime(*): 6424976209 - ScannerThreadsSysTime: 536917000 - ScannerThreadsUserTime: 1374790000 - ScannerThreadsVoluntaryContextSwitches: 10299 - TotalRawHdfsReadTime(*): 4639982818 - TotalReadThroughput: 6082098 - TotalTime: 41165847336 Instance 92442b2d9b7951de:381adaf0c8f9a989 (host=host-10-17-101-152.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:611/3.98 GB - AverageThreadTokens: 2.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 4085816 - PerHostPeakMemUsage: 775785752 - PrepareTime: 999998 - RowsProduced: 20011366 - TotalCpuTime: 19521984711 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 5981995350 - TotalStorageWaitTime: 7140994365 - TotalTime: 16170987340 DataStreamSender (dst_id=7) - BytesSent: 853323968 - InactiveTotalTime: 0 - NetworkThroughput(*): 681025239 - OverallThroughput: 154057527 - PeakMemoryUsage: 10912 - RowsReturned: 20011366 - SerializeBatchTime: 4269996591 - TotalTime: 5538995617 - TransmitDataRPCTime: 1252999036 - UncompressedRowBatchSize: 1596623416 HDFS_SCAN_NODE (id=2) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 611 Hdfs split stats (:<# splits>/): 0:611/3.98 GB Hdfs Read Thread Concurrency Bucket: 0:96.97% 1:3.03% 2:0% 3:0% File Formats: PARQUET/SNAPPY:611 - AverageHdfsReadThreadConcurrency: 0.030303030303030304 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 335063406 - BytesReadDataNodeCache: 0 - BytesReadLocal: 335063406 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 335063406 - DecompressionTime: 15999986 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 611 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 3960216 - PerReadThreadRawHdfsThroughput: 3317462224 - RemoteScanRanges: 0 - RowsRead: 20011366 - RowsReturned: 20011366 - RowsReturnedRate: 6548227 - ScanRangesComplete: 611 - ScannerThreadsInvoluntaryContextSwitches: 2740 - ScannerThreadsTotalWallClockTime: 16305987226 - MaterializeTupleTime(*): 7066994537 - ScannerThreadsSysTime: 523920000 - ScannerThreadsUserTime: 1432782000 - ScannerThreadsVoluntaryContextSwitches: 13473 - TotalRawHdfsReadTime(*): 100999916 - TotalReadThroughput: 19974041 - TotalTime: 3055997599 Fragment F01 Instance 92442b2d9b7951de:381adaf0c8f9a98a (host=host-10-17-101-153.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:610/3.97 GB Filter 0 arrival: 53s684ms - AverageThreadTokens: 2.1682242990654204 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 5216416 - PerHostPeakMemUsage: 798888872 - PrepareTime: 999998 - RowsProduced: 19988634 - TotalCpuTime: 63037766259 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 6118977322 - TotalStorageWaitTime: 52482805412 - TotalTime: 53511801558 DataStreamSender (dst_id=6) - BytesSent: 852410528 - InactiveTotalTime: 0 - NetworkThroughput(*): 788541781 - OverallThroughput: 166519570 - PeakMemoryUsage: 10912 - RowsReturned: 19988634 - SerializeBatchTime: 4029985076 - TotalTime: 5118981068 - TransmitDataRPCTime: 1080996021 - UncompressedRowBatchSize: 1594805262 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 610 Hdfs split stats (:<# splits>/): 0:610/3.97 GB Runtime filters: Only following filters arrived: , waited 1s004ms Hdfs Read Thread Concurrency Bucket: 0:90.65% 1:9.346% 2:0% 3:0% File Formats: PARQUET/SNAPPY:610 - AverageHdfsReadThreadConcurrency: 0.09345794392523364 - AverageScannerThreadConcurrency: 1.1682242990654206 - BytesRead: 334648896 - BytesReadDataNodeCache: 0 - BytesReadLocal: 334648896 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 334648896 - DecompressionTime: 11999955 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 610 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 5197312 - PerReadThreadRawHdfsThroughput: 109541780 - RemoteScanRanges: 0 - RowsRead: 19988634 - RowsReturned: 19988634 - RowsReturnedRate: 489931 - ScanRangesComplete: 610 - ScannerThreadsInvoluntaryContextSwitches: 4804 - ScannerThreadsTotalWallClockTime: 63037766259 - MaterializeTupleTime(*): 7924970643 - ScannerThreadsSysTime: 526919000 - ScannerThreadsUserTime: 1579759000 - ScannerThreadsVoluntaryContextSwitches: 9589 - TotalRawHdfsReadTime(*): 3054988647 - TotalReadThroughput: 5967365 - TotalTime: 40798848688 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 9843233 - Rows rejected: 0 - Rows total: 9843833 - TotalTime: 0 Instance 92442b2d9b7951de:381adaf0c8f9a98b (host=host-10-17-101-152.coe.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:611/3.98 GB - AverageThreadTokens: 2.2708333333333335 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 5807144 - PerHostPeakMemUsage: 775785752 - PrepareTime: 1999998 - RowsProduced: 20011366 - TotalCpuTime: 32236974730 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 9952992266 - TotalStorageWaitTime: 12596990046 - TotalTime: 23880981265 DataStreamSender (dst_id=6) - BytesSent: 853313740 - InactiveTotalTime: 0 - NetworkThroughput(*): 666651879 - OverallThroughput: 150363773 - PeakMemoryUsage: 10912 - RowsReturned: 20011366 - SerializeBatchTime: 4371996512 - TotalTime: 5674995495 - TransmitDataRPCTime: 1279999001 - UncompressedRowBatchSize: 1596623416 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 611 Hdfs split stats (:<# splits>/): 0:611/3.98 GB Runtime filters: Only following filters arrived: , waited 1s004ms Hdfs Read Thread Concurrency Bucket: 0:97.92% 1:2.083% 2:0% 3:0% File Formats: PARQUET/SNAPPY:611 - AverageHdfsReadThreadConcurrency: 0.020833333333333332 - AverageScannerThreadConcurrency: 1.2708333333333333 - BytesRead: 335063406 - BytesReadDataNodeCache: 0 - BytesReadLocal: 335063406 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 335063406 - DecompressionTime: 14999982 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 1 - NumDisksAccessed: 1 - NumRowGroups: 611 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 5788040 - PerReadThreadRawHdfsThroughput: 2659235618 - RemoteScanRanges: 0 - RowsRead: 20011366 - RowsReturned: 20011366 - RowsReturnedRate: 3009230 - ScanRangesComplete: 611 - ScannerThreadsInvoluntaryContextSwitches: 3197 - ScannerThreadsTotalWallClockTime: 30727975903 - MaterializeTupleTime(*): 16266987324 - ScannerThreadsSysTime: 511922000 - ScannerThreadsUserTime: 1659747000 - ScannerThreadsVoluntaryContextSwitches: 14848 - TotalRawHdfsReadTime(*): 125999894 - TotalReadThroughput: 13274690 - TotalTime: 6649994793 Filter 0 (1.00 MB) - InactiveTotalTime: 0 - Rows processed: 9996995 - Rows rejected: 0 - Rows total: 9997605 - TotalTime: 0