Query (id=aa4508fe4d6a8e3f:91477db763e7b4a5) Summary Session ID: f144afbb952c47fc:c43aeb5e60e26197 Session Type: BEESWAX Start Time: 2014-11-14 20:26:48.479114000 End Time: 2014-11-14 20:28:39.204790000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.1.0-cdh4-INTERNAL DEBUG (build cd0d73cfb0cc891782c7b34104fd8b79df70f356) User: jenkins Connected User: jenkins Delegated User: Network Address: 127.0.0.1:48741 Default Db: tpch Sql Statement: select s_name, count(*) as numwait from supplier, lineitem l1, orders, nation where s_suppkey = l1.l_suppkey and o_orderkey = l1.l_orderkey and o_orderstatus = 'F' and l1.l_receiptdate > l1.l_commitdate and exists ( select * from lineitem l2 where l2.l_orderkey = l1.l_orderkey and l2.l_suppkey <> l1.l_suppkey ) and not exists ( select * from lineitem l3 where l3.l_orderkey = l1.l_orderkey and l3.l_suppkey <> l1.l_suppkey and l3.l_receiptdate > l3.l_commitdate ) and s_nationkey = n_nationkey and n_name = 'SAUDI ARABIA' group by s_name order by numwait desc, s_name limit 100 Coordinator: a2124.halxg.cloudera.com:22000 Plan: ---------------- Estimated Per-Host Requirements: Memory=575.85MB VCores=4 F08:PLAN FRAGMENT [UNPARTITIONED] 21:MERGING-EXCHANGE [UNPARTITIONED] order by: count(*) DESC, s_name ASC limit: 100 hosts=3 per-host-mem=unavailable tuple-ids=9 row-size=42B cardinality=100 F07:PLAN FRAGMENT [HASH(s_name)] DATASTREAM SINK [FRAGMENT=F08, EXCHANGE=21, UNPARTITIONED] 12:TOP-N [LIMIT=100] | order by: count(*) DESC, s_name ASC | hosts=3 per-host-mem=4.10KB | tuple-ids=9 row-size=42B cardinality=100 | 20:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: s_name | hosts=3 per-host-mem=10.00MB | tuple-ids=8 row-size=42B cardinality=10388 | 19:EXCHANGE [HASH(s_name)] hosts=3 per-host-mem=0B tuple-ids=8 row-size=42B cardinality=10388 F05:PLAN FRAGMENT [HASH(l1.l_orderkey)] DATASTREAM SINK [FRAGMENT=F07, EXCHANGE=19, HASH(s_name)] 11:AGGREGATE | output: count(*) | group by: s_name | hosts=3 per-host-mem=10.00MB | tuple-ids=8 row-size=42B cardinality=10388 | 10:HASH JOIN [LEFT ANTI JOIN, PARTITIONED] | hash predicates: l1.l_orderkey = l3.l_orderkey | other join predicates: l3.l_suppkey != l1.l_suppkey | hosts=3 per-host-mem=14.27MB | tuple-ids=1,0,2,3 row-size=162B cardinality=600122 | |--18:EXCHANGE [HASH(l3.l_orderkey)] | hosts=3 per-host-mem=0B | tuple-ids=6 row-size=68B cardinality=600122 | 09:HASH JOIN [LEFT SEMI JOIN, PARTITIONED] | hash predicates: l1.l_orderkey = l2.l_orderkey | other join predicates: l2.l_suppkey != l1.l_suppkey | hosts=3 per-host-mem=33.58MB | tuple-ids=1,0,2,3 row-size=162B cardinality=600122 | |--17:EXCHANGE [HASH(l2.l_orderkey)] | hosts=3 per-host-mem=0B | tuple-ids=4 row-size=16B cardinality=6001215 | 16:EXCHANGE [HASH(l1.l_orderkey)] hosts=3 per-host-mem=0B tuple-ids=1,0,2,3 row-size=162B cardinality=600122 F06:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F05, EXCHANGE=18, HASH(l3.l_orderkey)] 05:SCAN HDFS [tpch.lineitem l3, RANDOM] partitions=1/1 files=1 size=718.94MB predicates: l3.l_receiptdate > l3.l_commitdate table stats: 6001215 rows total column stats: all hosts=3 per-host-mem=264.00MB tuple-ids=6 row-size=68B cardinality=600122 F04:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F05, EXCHANGE=17, HASH(l2.l_orderkey)] 04:SCAN HDFS [tpch.lineitem l2, RANDOM] partitions=1/1 files=1 size=718.94MB table stats: 6001215 rows total column stats: all hosts=3 per-host-mem=264.00MB tuple-ids=4 row-size=16B cardinality=6001215 F00:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F05, EXCHANGE=16, HASH(l1.l_orderkey)] 08:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: s_nationkey = n_nationkey | hosts=3 per-host-mem=28B | tuple-ids=1,0,2,3 row-size=162B cardinality=600122 | |--15:EXCHANGE [BROADCAST] | hosts=3 per-host-mem=0B | tuple-ids=3 row-size=25B cardinality=1 | 07:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: l1.l_orderkey = o_orderkey | hosts=3 per-host-mem=13.11MB | tuple-ids=1,0,2 row-size=137B cardinality=600122 | |--14:EXCHANGE [BROADCAST] | hosts=3 per-host-mem=0B | tuple-ids=2 row-size=25B cardinality=500000 | 06:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: l1.l_suppkey = s_suppkey | hosts=3 per-host-mem=472.66KB | tuple-ids=1,0 row-size=112B cardinality=600122 | |--13:EXCHANGE [BROADCAST] | hosts=3 per-host-mem=0B | tuple-ids=0 row-size=44B cardinality=10000 | 01:SCAN HDFS [tpch.lineitem l1, RANDOM] partitions=1/1 files=1 size=718.94MB predicates: l1.l_receiptdate > l1.l_commitdate table stats: 6001215 rows total column stats: all hosts=3 per-host-mem=264.00MB tuple-ids=1 row-size=68B cardinality=600122 F03:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=15, BROADCAST] 03:SCAN HDFS [tpch.nation, RANDOM] partitions=1/1 files=1 size=2.15KB predicates: n_name = 'SAUDI ARABIA' table stats: 25 rows total column stats: all hosts=3 per-host-mem=32.00MB tuple-ids=3 row-size=25B cardinality=1 F02:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=14, BROADCAST] 02:SCAN HDFS [tpch.orders, RANDOM] partitions=1/1 files=1 size=162.56MB predicates: o_orderstatus = 'F' table stats: 1500000 rows total column stats: all hosts=3 per-host-mem=176.00MB tuple-ids=2 row-size=25B cardinality=500000 F01:PLAN FRAGMENT [RANDOM] DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=13, BROADCAST] 00:SCAN HDFS [tpch.supplier, RANDOM] partitions=1/1 files=1 size=1.33MB table stats: 10000 rows total column stats: all hosts=3 per-host-mem=32.00MB tuple-ids=0 row-size=44B cardinality=10000 ---------------- Estimated Per-Host Mem: 603818298 Estimated Per-Host VCores: 4 Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------- 21:MERGING-EXCHANGE 1 694.171us 694.171us 300 100 0 -1.00 B UNPARTITIONED 12:TOP-N 3 2.875ms 3.402ms 300 100 20.00 KB 4.10 KB 20:AGGREGATE 3 1s110ms 1s111ms 411 10.39K 6.29 MB 10.00 MB FINALIZE 19:EXCHANGE 3 183.944us 198.813us 1.21K 10.39K 0 0 HASH(s_name) 11:AGGREGATE 3 371.973ms 519.24ms 1.21K 10.39K 8.92 MB 10.00 MB 10:HASH JOIN 3 2s069ms 2s159ms 13.91K 600.12K 838.05 MB 14.27 MB LEFT ANTI JOIN, PARTITIONED |--18:EXCHANGE 3 409.707ms 413.441ms 3.79M 600.12K 0 0 HASH(l3.l_orderkey) | 05:SCAN HDFS 3 1m20s 1m34s 3.79M 600.12K 24.30 MB 264.00 MB tpch.lineitem l3 09:HASH JOIN 3 41s041ms 41s195ms 73.09K 600.12K 838.05 MB 33.58 MB LEFT SEMI JOIN, PARTITIONED |--17:EXCHANGE 3 521.547ms 530.812ms 6.00M 6.00M 0 0 HASH(l2.l_orderkey) | 04:SCAN HDFS 3 1m19s 1m33s 6.00M 6.00M 24.12 MB 264.00 MB tpch.lineitem l2 16:EXCHANGE 3 8.868ms 10.867ms 75.87K 600.12K 0 0 HASH(l1.l_orderkey) 08:HASH JOIN 3 388.671ms 410.5ms 75.87K 600.12K 8.59 MB 28.00 B INNER JOIN, BROADCAST |--15:EXCHANGE 3 35.973us 41.837us 1 1 0 0 BROADCAST | 03:SCAN HDFS 1 7s453ms 7s453ms 1 1 41.00 KB 32.00 MB tpch.nation 07:HASH JOIN 3 42s495ms 43s984ms 1.83M 600.12K 566.03 MB 13.11 MB INNER JOIN, BROADCAST |--14:EXCHANGE 3 186.685ms 188.864ms 729.41K 500.00K 0 0 BROADCAST | 02:SCAN HDFS 2 38s291ms 56s436ms 729.41K 500.00K 16.14 MB 176.00 MB tpch.orders 06:HASH JOIN 3 5s841ms 8s681ms 3.79M 600.12K 12.43 MB 472.66 KB INNER JOIN, BROADCAST |--13:EXCHANGE 3 4.79ms 5.164ms 10.00K 10.00K 0 0 BROADCAST | 00:SCAN HDFS 1 7s837ms 7s837ms 10.00K 10.00K 2.08 MB 32.00 MB tpch.supplier 01:SCAN HDFS 3 43s272ms 54s934ms 3.79M 600.12K 83.99 MB 264.00 MB tpch.lineitem l1 Query Timeline Start execution: 110875 Planning finished: 105000098 Ready to start remote fragments: 107914475 Remote fragments started: 4212697781 Rows available: 110668844451 First row fetched: 110693201032 Unregister query: 110725639767 ImpalaServer - AsyncTotalTime: 0 - ClientFetchWaitTimer: 49050623 - InactiveTotalTime: 0 - RowMaterializationTimer: 315976 - TotalTime: 0 Execution Profile aa4508fe4d6a8e3f:91477db763e7b4a5 Fragment start latencies: count: 19, last: 0.263231, min: 0.257404, max: 1.11522, mean: 0.556599, stddev: 0.377176 Per Node Peak Memory Usage: a2124.halxg.cloudera.com:22002(1.91 GB) a2124.halxg.cloudera.com:22000(1.69 GB) a2124.halxg.cloudera.com:22001(1.72 GB) - AsyncTotalTime: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 110569505752 Coordinator Fragment F08 - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 49152 - PerHostPeakMemUsage: 0 - PrepareTime: 101210 - RowsProduced: 100 - TotalCpuTime: 110587567014 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 106456976699 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 802 - BlocksRecycled: 287 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 32385619968 - PeakMemoryUsage: 1678245888 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=21) - AsyncTotalTime: 0 - BytesReceived: 6706 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 239889 - FirstBatchArrivalWaitTime: 106455281751 - InactiveTotalTime: 106455283614 - MergeGetNext: 585711 - MergeGetNextBatch: 69076 - PeakMemoryUsage: 0 - RowsReturned: 300 - RowsReturnedRate: 2 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 106455977785 Averaged Fragment F07 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1m49s max:1m49s mean: 1m49s stddev:1.568ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 6619136 - PerHostPeakMemUsage: 1907465917 - PrepareTime: 1107624480 - RowsProduced: 100 - TotalCpuTime: 2771656824 - TotalNetworkReceiveTime: 107784905542 - TotalNetworkSendTime: 353746 - TotalStorageWaitTime: 0 - TotalTime: 108900072795 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 852 - BlocksRecycled: 279 - BufferedPins: 28 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 32385619968 - PeakMemoryUsage: 1768554504 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 9768852 - CompileTime: 1405278349 - InactiveTotalTime: 0 - LoadTime: 503706 - ModuleFileSize: 1955380 - OptimizationTime: 250216742 - PrepareTime: 244168467 - TotalTime: 1902605131 DataStreamSender (dst_id=21) - AsyncTotalTime: 0 - BytesSent: 2235 - InactiveTotalTime: 0 - NetworkThroughput(*): 8232482 - OverallThroughput: 13458954 - PeakMemoryUsage: 4096 - SerializeBatchTime: 94280 - ThriftTransmitTime(*): 273000 - TotalTime: 171011 - UncompressedRowBatchSize: 5404 SORT_NODE (id=12) - AsyncTotalTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 108898848599 AGGREGATION_NODE (id=20) - AsyncTotalTime: 0 - BuildTime: 256989 - GetNewBlockTime: 237576 - GetResultsTime: 319439 - HashBuckets: 32426 - InactiveTotalTime: 0 - LargestPartitionPercent: 1 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6598656 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 137 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 108895972823 - UnpinTime: 19260 EXCHANGE_NODE (id=19) - AsyncTotalTime: 0 - BytesReceived: 7478 - ConvertRowBatchTime: 98245 - DeserializeRowBatchTimer: 267163 - FirstBatchArrivalWaitTime: 106771670987 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 404 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 107785048514 Averaged Fragment F05 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:1m48s max:1m49s mean: 1m48s stddev:500.25ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 3 - AsyncTotalTime: 0 - AverageThreadTokens: 2.7856476513126958 - InactiveTotalTime: 0 - PeakMemoryUsage: 1526373672 - PerHostPeakMemUsage: 1907465917 - PrepareTime: 306054651 - RowsProduced: 404 - TotalCpuTime: 194463704428 - TotalNetworkReceiveTime: 257342454200 - TotalNetworkSendTime: 1224108 - TotalStorageWaitTime: 0 - TotalTime: 107213941577 CodeGen - AsyncTotalTime: 0 - CodegenTime: 6585545 - CompileTime: 1160805148 - InactiveTotalTime: 0 - LoadTime: 1313662 - ModuleFileSize: 1955380 - OptimizationTime: 384270020 - PrepareTime: 272948064 - TotalTime: 1821749725 DataStreamSender (dst_id=19) - AsyncTotalTime: 0 - BytesSent: 7478 - InactiveTotalTime: 0 - NetworkThroughput(*): 7949011 - OverallThroughput: 13948078 - PeakMemoryUsage: 36864 - SerializeBatchTime: 309527 - ThriftTransmitTime(*): 957287 - TotalTime: 543081 - UncompressedRowBatchSize: 21828 AGGREGATION_NODE (id=11) - AsyncTotalTime: 0 - BuildTime: 95519830 - GetNewBlockTime: 156820 - GetResultsTime: 418251 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 8799096 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 404 - RowsReturnedRate: 3 - SpilledPartitions: 0 - TotalTime: 107211887857 - UnpinTime: 14913 HASH_JOIN_NODE (id=10) - AsyncTotalTime: 0 - BuildPartitionTime: 808783063 - BuildRows: 1264432 - BuildRowsPartitioned: 1264432 - BuildTime: 743700277 - GetNewBlockTime: 29899480 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 4 - PartitionsCreated: 32 - PeakMemoryUsage: 803454976 - PinTime: 396934 - ProbeRows: 24363 - ProbeRowsPartitioned: 3256 - ProbeTime: 17885527 - RowsReturned: 4637 - RowsReturnedRate: 42 - SpilledPartitions: 4 - TotalTime: 106839914013 - UnpinTime: 892075 EXCHANGE_NODE (id=18) - AsyncTotalTime: 0 - BytesReceived: 52124038 - ConvertRowBatchTime: 303052418 - DeserializeRowBatchTimer: 863882975 - FirstBatchArrivalWaitTime: 4009797002 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1264432 - RowsReturnedRate: 13286 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95162154768 HASH_JOIN_NODE (id=9) - AsyncTotalTime: 0 - BuildPartitionTime: 521395542 - BuildRows: 2000405 - BuildRowsPartitioned: 2000405 - BuildTime: 1167873495 - GetNewBlockTime: 29426640 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 5 - PartitionsCreated: 32 - PeakMemoryUsage: 787354965 - PinTime: 552907 - ProbeRows: 25290 - ProbeRowsPartitioned: 3896 - ProbeTime: 19823580 - RowsReturned: 24363 - RowsReturnedRate: 228 - SpilledPartitions: 5 - TotalTime: 106659137997 - UnpinTime: 1581356 EXCHANGE_NODE (id=17) - AsyncTotalTime: 0 - BytesReceived: 30855787 - ConvertRowBatchTime: 453898517 - DeserializeRowBatchTimer: 261948740 - FirstBatchArrivalWaitTime: 4777283825 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 2000405 - RowsReturnedRate: 20923 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95604966258 EXCHANGE_NODE (id=16) - AsyncTotalTime: 0 - BytesReceived: 2622832 - ConvertRowBatchTime: 5850742 - DeserializeRowBatchTimer: 53010364 - FirstBatchArrivalWaitTime: 58407626754 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 25290 - RowsReturnedRate: 374 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 67425849974 Averaged Fragment F06 split sizes: min: 206.94 MB, max: 256.00 MB, avg: 239.65 MB, stddev: 23.13 MB completion times: min:1m8s max:1m36s mean: 1m22s stddev:11s305ms execution rates: min:2.66 MB/sec max:3.14 MB/sec mean:2.93 MB/sec stddev:208.01 KB/sec num instances: 3 - AsyncTotalTime: 0 - AverageThreadTokens: 2.9115568650983623 - InactiveTotalTime: 0 - PeakMemoryUsage: 25645056 - PerHostPeakMemUsage: 1838997125 - PrepareTime: 1107322965 - RowsProduced: 1264432 - TotalCpuTime: 159714801799 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 7381152 - TotalStorageWaitTime: 150114958582 - TotalTime: 83109690910 CodeGen - AsyncTotalTime: 0 - CodegenTime: 4685384 - CompileTime: 8841229 - InactiveTotalTime: 0 - LoadTime: 1379402 - ModuleFileSize: 1955380 - OptimizationTime: 231813754 - PrepareTime: 255445414 - TotalTime: 1000772921 DataStreamSender (dst_id=18) - AsyncTotalTime: 0 - BytesSent: 52124038 - InactiveTotalTime: 0 - NetworkThroughput(*): 28161056 - OverallThroughput: 16628524 - PeakMemoryUsage: 86016 - SerializeBatchTime: 1492856869 - ThriftTransmitTime(*): 1847391467 - TotalTime: 3143282527 - UncompressedRowBatchSize: 101171886 HDFS_SCAN_NODE (id=5) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.17718545751633977 - AverageScannerThreadConcurrency: 1.9437863108206246 - BytesRead: 251289064 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251289064 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251289064 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25485312 - PerReadThreadRawHdfsThroughput: 16732861 - RemoteScanRanges: 0 - RowsRead: 2000405 - RowsReturned: 1264432 - RowsReturnedRate: 15869 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 519 - ScannerThreadsTotalWallClockTime: 159714808483 - DelimiterParseTime: 3002711142 - MaterializeTupleTime(*): 364831295 - ScannerThreadsSysTime: 4998333 - ScannerThreadsUserTime: 3383817666 - ScannerThreadsVoluntaryContextSwitches: 39 - TotalRawHdfsReadTime(*): 15475828899 - TotalReadThroughput: 3054372 - TotalTime: 80215303719 Averaged Fragment F04 split sizes: min: 206.94 MB, max: 256.00 MB, avg: 239.65 MB, stddev: 23.13 MB completion times: min:1m9s max:1m36s mean: 1m22s stddev:11s036ms execution rates: min:2.66 MB/sec max:3.13 MB/sec mean:2.93 MB/sec stddev:200.79 KB/sec num instances: 3 - AsyncTotalTime: 0 - AverageThreadTokens: 2.9260459444904616 - InactiveTotalTime: 0 - PeakMemoryUsage: 25367824 - PerHostPeakMemUsage: 1838997125 - PrepareTime: 276831477 - RowsProduced: 2000405 - TotalCpuTime: 159941864715 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 3783782 - TotalStorageWaitTime: 150433244339 - TotalTime: 82476302935 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2856334 - CompileTime: 4713678 - InactiveTotalTime: 0 - LoadTime: 1335286 - ModuleFileSize: 1955380 - OptimizationTime: 203653994 - PrepareTime: 261507365 - TotalTime: 482472990 DataStreamSender (dst_id=17) - AsyncTotalTime: 0 - BytesSent: 30855787 - InactiveTotalTime: 0 - NetworkThroughput(*): 29713866 - OverallThroughput: 11150145 - PeakMemoryUsage: 114576 - SerializeBatchTime: 1093600134 - ThriftTransmitTime(*): 1037344017 - TotalTime: 2782065136 - UncompressedRowBatchSize: 56023080 HDFS_SCAN_NODE (id=4) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.1963759780385882 - AverageScannerThreadConcurrency: 1.9379320596603542 - BytesRead: 251289064 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251289064 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251289064 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25288704 - PerReadThreadRawHdfsThroughput: 15534814 - RemoteScanRanges: 0 - RowsRead: 2000405 - RowsReturned: 2000405 - RowsReturnedRate: 25262 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 488 - ScannerThreadsTotalWallClockTime: 159941870067 - DelimiterParseTime: 2842493577 - MaterializeTupleTime(*): 308379198 - ScannerThreadsSysTime: 7664333 - ScannerThreadsUserTime: 3152853333 - ScannerThreadsVoluntaryContextSwitches: 39 - TotalRawHdfsReadTime(*): 16158916719 - TotalReadThroughput: 3059152 - TotalTime: 79686670027 Averaged Fragment F00 split sizes: min: 206.94 MB, max: 256.00 MB, avg: 239.65 MB, stddev: 23.13 MB completion times: min:1m23s max:1m46s mean: 1m34s stddev:9s324ms execution rates: min:2.41 MB/sec max:2.77 MB/sec mean:2.55 MB/sec stddev:159.04 KB/sec num instances: 3 - AsyncTotalTime: 0 - AverageThreadTokens: 3.3346508860213837 - InactiveTotalTime: 0 - PeakMemoryUsage: 675083621 - PerHostPeakMemUsage: 1907465917 - PrepareTime: 343235787 - RowsProduced: 25290 - TotalCpuTime: 221943515887 - TotalNetworkReceiveTime: 71906318083 - TotalNetworkSendTime: 1974789 - TotalStorageWaitTime: 76252769457 - TotalTime: 93969809881 CodeGen - AsyncTotalTime: 0 - CodegenTime: 4812933 - CompileTime: 42470221 - InactiveTotalTime: 0 - LoadTime: 1339763 - ModuleFileSize: 1955380 - OptimizationTime: 456163761 - PrepareTime: 296573265 - TotalTime: 799396155 DataStreamSender (dst_id=16) - AsyncTotalTime: 0 - BytesSent: 2622832 - InactiveTotalTime: 0 - NetworkThroughput(*): 24185430 - OverallThroughput: 13862474 - PeakMemoryUsage: 151552 - SerializeBatchTime: 93575184 - ThriftTransmitTime(*): 108495914 - TotalTime: 190458197 - UncompressedRowBatchSize: 5339865 HASH_JOIN_NODE (id=8) - AsyncTotalTime: 0 - BuildPartitionTime: 15403 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 10300 - GetNewBlockTime: 1024081 - HashBuckets: 2 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 8972986 - PinTime: 0 - ProbeRows: 609637 - ProbeRowsPartitioned: 0 - ProbeTime: 44598961 - RowsReturned: 25290 - RowsReturnedRate: 269 - SpilledPartitions: 0 - TotalTime: 93808417209 - UnpinTime: 17484 EXCHANGE_NODE (id=15) - AsyncTotalTime: 0 - BytesReceived: 44 - ConvertRowBatchTime: 2892 - DeserializeRowBatchTimer: 21134 - FirstBatchArrivalWaitTime: 7178728529 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 7179708981 HASH_JOIN_NODE (id=7) - AsyncTotalTime: 0 - BuildPartitionTime: 341033856 - BuildRows: 729413 - BuildRowsPartitioned: 729413 - BuildTime: 371373605 - GetNewBlockTime: 30305938 - HashBuckets: 2097152 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 593526784 - PinTime: 0 - ProbeRows: 1264432 - ProbeRowsPartitioned: 0 - ProbeTime: 233606017 - RowsReturned: 609637 - RowsReturnedRate: 6536 - SpilledPartitions: 0 - TotalTime: 93422030092 - UnpinTime: 20199 EXCHANGE_NODE (id=14) - AsyncTotalTime: 0 - BytesReceived: 13357177 - ConvertRowBatchTime: 167737178 - DeserializeRowBatchTimer: 332496759 - FirstBatchArrivalWaitTime: 4556520171 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 729413 - RowsReturnedRate: 12850 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 56759656603 HASH_JOIN_NODE (id=6) - AsyncTotalTime: 0 - BuildPartitionTime: 4560576 - BuildRows: 10000 - BuildRowsPartitioned: 10000 - BuildTime: 6562272 - GetNewBlockTime: 491324 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 13033472 - PinTime: 0 - ProbeRows: 1264432 - ProbeRowsPartitioned: 0 - ProbeTime: 347574057 - RowsReturned: 1264432 - RowsReturnedRate: 24855 - SpilledPartitions: 0 - TotalTime: 51815459134 - UnpinTime: 17344 EXCHANGE_NODE (id=13) - AsyncTotalTime: 0 - BytesReceived: 186983 - ConvertRowBatchTime: 2527103 - DeserializeRowBatchTimer: 4732353 - FirstBatchArrivalWaitTime: 5708960149 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 10000 - RowsReturnedRate: 1227 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 8144856963 HDFS_SCAN_NODE (id=1) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.17905781197635542 - AverageScannerThreadConcurrency: 1.6316650808950934 - BytesRead: 251289064 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251288381 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251288381 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 76825941 - PerReadThreadRawHdfsThroughput: 16108551 - RemoteScanRanges: 0 - RowsRead: 2000405 - RowsReturned: 1264432 - RowsReturnedRate: 30139 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 618 - ScannerThreadsTotalWallClockTime: 151669909102 - DelimiterParseTime: 3135971106 - MaterializeTupleTime(*): 396500195 - ScannerThreadsSysTime: 2998666 - ScannerThreadsUserTime: 4190362000 - ScannerThreadsVoluntaryContextSwitches: 33 - TotalRawHdfsReadTime(*): 15940745434 - TotalReadThroughput: 2648652 - TotalTime: 43272744353 Averaged Fragment F03 split sizes: min: 2.15 KB, max: 2.15 KB, avg: 2.15 KB, stddev: 0 completion times: min:7s392ms max:7s392ms mean: 7s392ms stddev:0ns execution rates: min:297.00 B/sec max:297.00 B/sec mean:297.00 B/sec stddev:0.00 /sec num instances: 1 - AsyncTotalTime: 0 - AverageThreadTokens: 1.875 - InactiveTotalTime: 0 - PeakMemoryUsage: 66544 - PerHostPeakMemUsage: 120999472 - PrepareTime: 271264033 - RowsProduced: 1 - TotalCpuTime: 9403777724 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 2001406 - TotalStorageWaitTime: 5399470499 - TotalTime: 7416248118 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2829575 - CompileTime: 6439215 - InactiveTotalTime: 0 - LoadTime: 1370031 - ModuleFileSize: 1955380 - OptimizationTime: 240108859 - PrepareTime: 224298760 - TotalTime: 473619911 DataStreamSender (dst_id=15) - AsyncTotalTime: 0 - BytesSent: 144 - InactiveTotalTime: 0 - NetworkThroughput(*): 41818 - OverallThroughput: 67209 - PeakMemoryUsage: 16368 - SerializeBatchTime: 23044 - ThriftTransmitTime(*): 3443482 - TotalTime: 2142565 - UncompressedRowBatchSize: 144 HDFS_SCAN_NODE (id=3) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 2199 - BytesReadDataNodeCache: 0 - BytesReadLocal: 2199 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2199 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 41984 - PerReadThreadRawHdfsThroughput: 967967 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 7140281624 - DelimiterParseTime: 20059 - MaterializeTupleTime(*): 4900 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 4 - TotalRawHdfsReadTime(*): 2271771 - TotalReadThroughput: 293 - TotalTime: 7453715653 Averaged Fragment F02 split sizes: min: 34.56 MB, max: 128.00 MB, avg: 81.28 MB, stddev: 46.72 MB completion times: min:20s311ms max:57s057ms mean: 38s684ms stddev:18s373ms execution rates: min:1.70 MB/sec max:2.24 MB/sec mean:1.97 MB/sec stddev:277.50 KB/sec num instances: 2 - AsyncTotalTime: 0 - AverageThreadTokens: 1.9834189131365005 - InactiveTotalTime: 0 - PeakMemoryUsage: 16916480 - PerHostPeakMemUsage: 1266344208 - PrepareTime: 283764481 - RowsProduced: 364706 - TotalCpuTime: 41574179605 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1285617 - TotalStorageWaitTime: 35698539874 - TotalTime: 38587955632 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2953988 - CompileTime: 4429213 - InactiveTotalTime: 0 - LoadTime: 1343545 - ModuleFileSize: 1955380 - OptimizationTime: 370408860 - PrepareTime: 277379004 - TotalTime: 655561309 DataStreamSender (dst_id=14) - AsyncTotalTime: 0 - BytesSent: 20035777 - InactiveTotalTime: 0 - NetworkThroughput(*): 21933285 - OverallThroughput: 64841808 - PeakMemoryUsage: 12288 - SerializeBatchTime: 283051130 - ThriftTransmitTime(*): 859307119 - TotalTime: 293276100 - UncompressedRowBatchSize: 40486735 HDFS_SCAN_NODE (id=2) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.167146017699115 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 85226592 - BytesReadDataNodeCache: 0 - BytesReadLocal: 85226592 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 85226592 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 16896000 - PerReadThreadRawHdfsThroughput: 16225991 - RemoteScanRanges: 0 - RowsRead: 750000 - RowsReturned: 364706 - RowsReturnedRate: 8938 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 103 - ScannerThreadsTotalWallClockTime: 38304967589 - DelimiterParseTime: 704762532 - MaterializeTupleTime(*): 67738819 - ScannerThreadsSysTime: 999500 - ScannerThreadsUserTime: 782880500 - ScannerThreadsVoluntaryContextSwitches: 14 - TotalRawHdfsReadTime(*): 5019371208 - TotalReadThroughput: 2061114 - TotalTime: 38291632591 Averaged Fragment F01 split sizes: min: 1.33 MB, max: 1.33 MB, avg: 1.33 MB, stddev: 0 completion times: min:7s783ms max:7s783ms mean: 7s783ms stddev:0ns execution rates: min:175.54 KB/sec max:175.54 KB/sec mean:175.54 KB/sec stddev:0.00 /sec num instances: 1 - AsyncTotalTime: 0 - AverageThreadTokens: 1.9375 - InactiveTotalTime: 0 - PeakMemoryUsage: 2244608 - PerHostPeakMemUsage: 125284944 - PrepareTime: 260327422 - RowsProduced: 10000 - TotalCpuTime: 10493117860 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 2674013 - TotalStorageWaitTime: 5136317846 - TotalTime: 7851134868 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2444843 - CompileTime: 6238460 - InactiveTotalTime: 0 - LoadTime: 1469385 - ModuleFileSize: 1955380 - OptimizationTime: 183247851 - PrepareTime: 253613192 - TotalTime: 447407801 DataStreamSender (dst_id=13) - AsyncTotalTime: 0 - BytesSent: 560961 - InactiveTotalTime: 0 - NetworkThroughput(*): 22574216 - OverallThroughput: 46480024 - PeakMemoryUsage: 12288 - SerializeBatchTime: 9240353 - ThriftTransmitTime(*): 24849633 - TotalTime: 12068862 - UncompressedRowBatchSize: 1620132 HDFS_SCAN_NODE (id=0) - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 1399184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1399184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1399184 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 2183168 - PerReadThreadRawHdfsThroughput: 73743009 - RemoteScanRanges: 0 - RowsRead: 10000 - RowsReturned: 10000 - RowsReturnedRate: 1275 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 7587448862 - DelimiterParseTime: 10012517 - MaterializeTupleTime(*): 1756773 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 11998000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 18973785 - TotalReadThroughput: 174898 - TotalTime: 7837430502 Fragment F07 Instance aa4508fe4d6a8e3f:91477db763e7b4a8 (host=a2124.halxg.cloudera.com:22002) - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 6619136 - PerHostPeakMemUsage: 2055714264 - PrepareTime: 1108315827 - RowsProduced: 100 - TotalCpuTime: 2762574972 - TotalNetworkReceiveTime: 107792608593 - TotalNetworkSendTime: 358827 - TotalStorageWaitTime: 0 - TotalTime: 108907766604 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 902 - BlocksRecycled: 187 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 32385619968 - PeakMemoryUsage: 1888223248 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 10397260 - CompileTime: 1408069059 - InactiveTotalTime: 0 - LoadTime: 540100 - ModuleFileSize: 1955380 - OptimizationTime: 238710724 - PrepareTime: 243509167 - TotalTime: 1893445864 DataStreamSender (dst_id=21) - AsyncTotalTime: 0 - BytesSent: 2243 - InactiveTotalTime: 0 - NetworkThroughput(*): 7834275 - OverallThroughput: 14658310 - PeakMemoryUsage: 4096 - SerializeBatchTime: 76569 - ThriftTransmitTime(*): 286306 - TotalTime: 153019 - UncompressedRowBatchSize: 5404 SORT_NODE (id=12) - AsyncTotalTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 108906331421 AGGREGATION_NODE (id=20) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 241941 - GetNewBlockTime: 242576 - GetResultsTime: 283210 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 1 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6598656 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 144 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 108903871074 - UnpinTime: 19796 EXCHANGE_NODE (id=19) - AsyncTotalTime: 0 - BytesReceived: 7760 - ConvertRowBatchTime: 80763 - DeserializeRowBatchTimer: 291539 - FirstBatchArrivalWaitTime: 106780114022 - InactiveTotalTime: 107792565282 - PeakMemoryUsage: 0 - RowsReturned: 423 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 107792730162 Instance aa4508fe4d6a8e3f:91477db763e7b4a9 (host=a2124.halxg.cloudera.com:22001) - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 6619136 - PerHostPeakMemUsage: 1850837872 - PrepareTime: 1108234711 - RowsProduced: 100 - TotalCpuTime: 2771238406 - TotalNetworkReceiveTime: 107786517170 - TotalNetworkSendTime: 344713 - TotalStorageWaitTime: 0 - TotalTime: 108901795923 BlockMgr - AsyncTotalTime: 0 - BlockWritesOutstanding: 0 - BlocksCreated: 802 - BlocksRecycled: 371 - BufferedPins: 56 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 32385619968 - PeakMemoryUsage: 1648885760 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - AsyncTotalTime: 0 - CodegenTime: 10917224 - CompileTime: 1387240749 - InactiveTotalTime: 0 - LoadTime: 19927 - ModuleFileSize: 1955380 - OptimizationTime: 266962020 - PrepareTime: 235143577 - TotalTime: 1890685101 DataStreamSender (dst_id=21) - AsyncTotalTime: 0 - BytesSent: 2282 - InactiveTotalTime: 0 - NetworkThroughput(*): 9204359 - OverallThroughput: 15442500 - PeakMemoryUsage: 4096 - SerializeBatchTime: 76800 - ThriftTransmitTime(*): 247926 - TotalTime: 147774 - UncompressedRowBatchSize: 5404 SORT_NODE (id=12) - AsyncTotalTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 108900739899 AGGREGATION_NODE (id=20) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 229937 - GetNewBlockTime: 285981 - GetResultsTime: 295092 - HashBuckets: 31744 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6598656 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 135 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 108897975677 - UnpinTime: 22812 EXCHANGE_NODE (id=19) - AsyncTotalTime: 0 - BytesReceived: 7387 - ConvertRowBatchTime: 103893 - DeserializeRowBatchTimer: 251076 - FirstBatchArrivalWaitTime: 106771778018 - InactiveTotalTime: 107786480215 - PeakMemoryUsage: 0 - RowsReturned: 399 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 107786668356 Instance aa4508fe4d6a8e3f:91477db763e7b4a7 (host=a2124.halxg.cloudera.com:22000) - AsyncTotalTime: 0 - AverageThreadTokens: 1.0 - InactiveTotalTime: 0 - PeakMemoryUsage: 6619136 - PerHostPeakMemUsage: 1815845616 - PrepareTime: 1106322903 - RowsProduced: 100 - TotalCpuTime: 2781157094 - TotalNetworkReceiveTime: 107775590865 - TotalNetworkSendTime: 357700 - TotalStorageWaitTime: 0 - TotalTime: 108890655859 CodeGen - AsyncTotalTime: 0 - CodegenTime: 7992074 - CompileTime: 1420525241 - InactiveTotalTime: 0 - LoadTime: 951093 - ModuleFileSize: 1955380 - OptimizationTime: 244977482 - PrepareTime: 253852657 - TotalTime: 1923684430 DataStreamSender (dst_id=21) - AsyncTotalTime: 0 - BytesSent: 2181 - InactiveTotalTime: 0 - NetworkThroughput(*): 7658812 - OverallThroughput: 10276054 - PeakMemoryUsage: 4096 - SerializeBatchTime: 129473 - ThriftTransmitTime(*): 284770 - TotalTime: 212241 - UncompressedRowBatchSize: 5404 SORT_NODE (id=12) - AsyncTotalTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 108889474477 AGGREGATION_NODE (id=20) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 299089 - GetNewBlockTime: 184171 - GetResultsTime: 380017 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 6598656 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 132 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 108886071720 - UnpinTime: 15173 EXCHANGE_NODE (id=19) - AsyncTotalTime: 0 - BytesReceived: 7289 - ConvertRowBatchTime: 110080 - DeserializeRowBatchTimer: 258876 - FirstBatchArrivalWaitTime: 106763120923 - InactiveTotalTime: 107775548213 - PeakMemoryUsage: 0 - RowsReturned: 390 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 107775747026 Fragment F05 Instance aa4508fe4d6a8e3f:91477db763e7b4ac (host=a2124.halxg.cloudera.com:22001) - AsyncTotalTime: 0 - AverageThreadTokens: 2.771689497716895 - InactiveTotalTime: 0 - PeakMemoryUsage: 1257313152 - PerHostPeakMemUsage: 1850837872 - PrepareTime: 276285301 - RowsProduced: 411 - TotalCpuTime: 193969089571 - TotalNetworkReceiveTime: 257445301810 - TotalNetworkSendTime: 1353273 - TotalStorageWaitTime: 0 - TotalTime: 107857368282 CodeGen - AsyncTotalTime: 0 - CodegenTime: 6422051 - CompileTime: 1187305900 - InactiveTotalTime: 0 - LoadTime: 1262847 - ModuleFileSize: 1955380 - OptimizationTime: 388140326 - PrepareTime: 242383947 - TotalTime: 1821706538 DataStreamSender (dst_id=19) - AsyncTotalTime: 0 - BytesSent: 7784 - InactiveTotalTime: 0 - NetworkThroughput(*): 6982424 - OverallThroughput: 16022131 - PeakMemoryUsage: 36864 - SerializeBatchTime: 293673 - ThriftTransmitTime(*): 1114799 - TotalTime: 485828 - UncompressedRowBatchSize: 22206 AGGREGATION_NODE (id=11) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 273682891 - GetNewBlockTime: 161942 - GetResultsTime: 387968 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 7998496 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 411 - RowsReturnedRate: 3 - SpilledPartitions: 0 - TotalTime: 107855642358 - UnpinTime: 15014 HASH_JOIN_NODE (id=10) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously, Spilled - AsyncTotalTime: 96858472834 - BuildPartitionTime: 808473557 - BuildRows: 1263636 - BuildRowsPartitioned: 1263636 - BuildTime: 781142616 - GetNewBlockTime: 27394120 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 13 - PartitionsCreated: 32 - PeakMemoryUsage: 652853248 - PinTime: 1190802 - ProbeRows: 24084 - ProbeRowsPartitioned: 9770 - ProbeTime: 25360337 - RowsReturned: 11099 - RowsReturnedRate: 103 - SpilledPartitions: 13 - TotalTime: 107336617600 - UnpinTime: 2643847 EXCHANGE_NODE (id=18) - AsyncTotalTime: 0 - BytesReceived: 52086652 - ConvertRowBatchTime: 306225293 - DeserializeRowBatchTimer: 872748382 - FirstBatchArrivalWaitTime: 4009605725 - InactiveTotalTime: 94749907031 - PeakMemoryUsage: 0 - RowsReturned: 1263636 - RowsReturnedRate: 13278 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95163348712 HASH_JOIN_NODE (id=9) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously, Spilled - AsyncTotalTime: 97110611254 - BuildPartitionTime: 537994899 - BuildRows: 1998412 - BuildRowsPartitioned: 1998412 - BuildTime: 1225454153 - GetNewBlockTime: 28759661 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 15 - PartitionsCreated: 32 - PeakMemoryUsage: 604553216 - PinTime: 1658721 - ProbeRows: 24987 - ProbeRowsPartitioned: 11689 - ProbeTime: 31710082 - RowsReturned: 24084 - RowsReturnedRate: 225 - SpilledPartitions: 15 - TotalTime: 106872362759 - UnpinTime: 4712057 EXCHANGE_NODE (id=17) - AsyncTotalTime: 0 - BytesReceived: 30830175 - ConvertRowBatchTime: 446119331 - DeserializeRowBatchTimer: 257943174 - FirstBatchArrivalWaitTime: 4777511313 - InactiveTotalTime: 95074608343 - PeakMemoryUsage: 0 - RowsReturned: 1998412 - RowsReturnedRate: 20906 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95588376765 EXCHANGE_NODE (id=16) - AsyncTotalTime: 0 - BytesReceived: 2591866 - ConvertRowBatchTime: 7339833 - DeserializeRowBatchTimer: 51583179 - FirstBatchArrivalWaitTime: 58407142061 - InactiveTotalTime: 67530633260 - PeakMemoryUsage: 0 - RowsReturned: 24987 - RowsReturnedRate: 369 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 67541500456 Instance aa4508fe4d6a8e3f:91477db763e7b4ab (host=a2124.halxg.cloudera.com:22002) - AsyncTotalTime: 0 - AverageThreadTokens: 2.792626728110599 - InactiveTotalTime: 0 - PeakMemoryUsage: 1670997448 - PerHostPeakMemUsage: 2055714264 - PrepareTime: 323567056 - RowsProduced: 401 - TotalCpuTime: 194819620345 - TotalNetworkReceiveTime: 257261433286 - TotalNetworkSendTime: 992962 - TotalStorageWaitTime: 0 - TotalTime: 106902200098 CodeGen - AsyncTotalTime: 0 - CodegenTime: 6537010 - CompileTime: 1164084438 - InactiveTotalTime: 0 - LoadTime: 1217779 - ModuleFileSize: 1955380 - OptimizationTime: 357377502 - PrepareTime: 292537565 - TotalTime: 1816459760 DataStreamSender (dst_id=19) - AsyncTotalTime: 0 - BytesSent: 7345 - InactiveTotalTime: 0 - NetworkThroughput(*): 9330597 - OverallThroughput: 14045776 - PeakMemoryUsage: 36864 - SerializeBatchTime: 348371 - ThriftTransmitTime(*): 787195 - TotalTime: 522933 - UncompressedRowBatchSize: 21666 AGGREGATION_NODE (id=11) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 6099354 - GetNewBlockTime: 143593 - GetResultsTime: 398718 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 1 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 9352896 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 401 - RowsReturnedRate: 3 - SpilledPartitions: 0 - TotalTime: 106900254488 - UnpinTime: 14590 HASH_JOIN_NODE (id=10) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 97219570980 - BuildPartitionTime: 819444335 - BuildRows: 1265391 - BuildRowsPartitioned: 1265391 - BuildTime: 768320647 - GetNewBlockTime: 26637079 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 878755840 - PinTime: 0 - ProbeRows: 24745 - ProbeRowsPartitioned: 0 - ProbeTime: 13883702 - RowsReturned: 1427 - RowsReturnedRate: 13 - SpilledPartitions: 0 - TotalTime: 106599372589 - UnpinTime: 15482 EXCHANGE_NODE (id=18) - AsyncTotalTime: 0 - BytesReceived: 52154161 - ConvertRowBatchTime: 306059593 - DeserializeRowBatchTimer: 851325157 - FirstBatchArrivalWaitTime: 4015856426 - InactiveTotalTime: 94746416965 - PeakMemoryUsage: 0 - RowsReturned: 1265391 - RowsReturnedRate: 13297 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95159141672 HASH_JOIN_NODE (id=9) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 97600044420 - BuildPartitionTime: 515317723 - BuildRows: 2003808 - BuildRowsPartitioned: 2003808 - BuildTime: 1157935604 - GetNewBlockTime: 27554805 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 878755840 - PinTime: 0 - ProbeRows: 25707 - ProbeRowsPartitioned: 0 - ProbeTime: 13516524 - RowsReturned: 24745 - RowsReturnedRate: 232 - SpilledPartitions: 0 - TotalTime: 106560881863 - UnpinTime: 17070 EXCHANGE_NODE (id=17) - AsyncTotalTime: 0 - BytesReceived: 30908268 - ConvertRowBatchTime: 463633839 - DeserializeRowBatchTimer: 258947559 - FirstBatchArrivalWaitTime: 4783230842 - InactiveTotalTime: 95087110217 - PeakMemoryUsage: 0 - RowsReturned: 2003808 - RowsReturnedRate: 20956 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95617922253 EXCHANGE_NODE (id=16) - AsyncTotalTime: 0 - BytesReceived: 2666914 - ConvertRowBatchTime: 5231888 - DeserializeRowBatchTimer: 54151718 - FirstBatchArrivalWaitTime: 58414369028 - InactiveTotalTime: 67339328673 - PeakMemoryUsage: 0 - RowsReturned: 25707 - RowsReturnedRate: 381 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 67347248280 Instance aa4508fe4d6a8e3f:91477db763e7b4aa (host=a2124.halxg.cloudera.com:22000) - AsyncTotalTime: 0 - AverageThreadTokens: 2.792626728110599 - InactiveTotalTime: 0 - PeakMemoryUsage: 1650810416 - PerHostPeakMemUsage: 1815845616 - PrepareTime: 318311597 - RowsProduced: 400 - TotalCpuTime: 194602403370 - TotalNetworkReceiveTime: 257320627506 - TotalNetworkSendTime: 1326091 - TotalStorageWaitTime: 0 - TotalTime: 106882256352 CodeGen - AsyncTotalTime: 0 - CodegenTime: 6797574 - CompileTime: 1131025107 - InactiveTotalTime: 0 - LoadTime: 1460361 - ModuleFileSize: 1955380 - OptimizationTime: 407292233 - PrepareTime: 283922681 - TotalTime: 1827082879 DataStreamSender (dst_id=19) - AsyncTotalTime: 0 - BytesSent: 7307 - InactiveTotalTime: 0 - NetworkThroughput(*): 7534014 - OverallThroughput: 11776328 - PeakMemoryUsage: 36864 - SerializeBatchTime: 286539 - ThriftTransmitTime(*): 969868 - TotalTime: 620482 - UncompressedRowBatchSize: 21612 AGGREGATION_NODE (id=11) ExecOption: Codegen Enabled - AsyncTotalTime: 0 - BuildTime: 6777246 - GetNewBlockTime: 164926 - GetResultsTime: 468068 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 1 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 9045896 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 400 - RowsReturnedRate: 3 - SpilledPartitions: 0 - TotalTime: 106879766726 - UnpinTime: 15137 HASH_JOIN_NODE (id=10) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 97074175187 - BuildPartitionTime: 798431299 - BuildRows: 1264269 - BuildRowsPartitioned: 1264269 - BuildTime: 681637568 - GetNewBlockTime: 35667243 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 878755840 - PinTime: 0 - ProbeRows: 24260 - ProbeRowsPartitioned: 0 - ProbeTime: 14412543 - RowsReturned: 1385 - RowsReturnedRate: 12 - SpilledPartitions: 0 - TotalTime: 106583751851 - UnpinTime: 16897 EXCHANGE_NODE (id=18) - AsyncTotalTime: 0 - BytesReceived: 52131301 - ConvertRowBatchTime: 296872368 - DeserializeRowBatchTimer: 867575386 - FirstBatchArrivalWaitTime: 4003928857 - InactiveTotalTime: 94761019210 - PeakMemoryUsage: 0 - RowsReturned: 1264269 - RowsReturnedRate: 13285 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95163973920 HASH_JOIN_NODE (id=9) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 97528221591 - BuildPartitionTime: 510874006 - BuildRows: 1998995 - BuildRowsPartitioned: 1998995 - BuildTime: 1120230730 - GetNewBlockTime: 31965455 - HashBuckets: 4194304 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 878755840 - PinTime: 0 - ProbeRows: 25177 - ProbeRowsPartitioned: 0 - ProbeTime: 14244135 - RowsReturned: 24260 - RowsReturnedRate: 227 - SpilledPartitions: 0 - TotalTime: 106544169369 - UnpinTime: 14942 EXCHANGE_NODE (id=17) - AsyncTotalTime: 0 - BytesReceived: 30828919 - ConvertRowBatchTime: 451942383 - DeserializeRowBatchTimer: 268955487 - FirstBatchArrivalWaitTime: 4771109320 - InactiveTotalTime: 95088538179 - PeakMemoryUsage: 0 - RowsReturned: 1998995 - RowsReturnedRate: 20908 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 95608599758 EXCHANGE_NODE (id=16) - AsyncTotalTime: 0 - BytesReceived: 2609716 - ConvertRowBatchTime: 4980507 - DeserializeRowBatchTimer: 53296196 - FirstBatchArrivalWaitTime: 58401369175 - InactiveTotalTime: 67380982423 - PeakMemoryUsage: 0 - RowsReturned: 25177 - RowsReturnedRate: 373 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 67388801187 Fragment F06 Instance aa4508fe4d6a8e3f:91477db763e7b4ae (host=a2124.halxg.cloudera.com:22002) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.9690721649484537 - InactiveTotalTime: 0 - PeakMemoryUsage: 25645056 - PerHostPeakMemUsage: 1850307888 - PrepareTime: 1106776401 - RowsProduced: 1354407 - TotalCpuTime: 190986921501 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 8362033 - TotalStorageWaitTime: 180867316167 - TotalTime: 97261303860 CodeGen - AsyncTotalTime: 0 - CodegenTime: 4256268 - CompileTime: 4667339 - InactiveTotalTime: 0 - LoadTime: 1274175 - ModuleFileSize: 1955380 - OptimizationTime: 219171583 - PrepareTime: 227553477 - TotalTime: 983674225 DataStreamSender (dst_id=18) - AsyncTotalTime: 0 - BytesSent: 55066464 - InactiveTotalTime: 0 - NetworkThroughput(*): 27931423 - OverallThroughput: 15947007 - PeakMemoryUsage: 86016 - SerializeBatchTime: 1651841239 - ThriftTransmitTime(*): 1971487938 - TotalTime: 3453090703 - UncompressedRowBatchSize: 108371120 HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:79.69% 1:20.31% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.203125 - AverageScannerThreadConcurrency: 1.9895833333333333 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268437504 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268437504 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25485312 - PerReadThreadRawHdfsThroughput: 13324913 - RemoteScanRanges: 0 - RowsRead: 2142543 - RowsReturned: 1354407 - RowsReturnedRate: 14399 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 620 - ScannerThreadsTotalWallClockTime: 190986926695 - DelimiterParseTime: 3303730588 - MaterializeTupleTime(*): 396888913 - ScannerThreadsSysTime: 4998000 - ScannerThreadsUserTime: 3709435000 - ScannerThreadsVoluntaryContextSwitches: 43 - TotalRawHdfsReadTime(*): 20145534742 - TotalReadThroughput: 2767329 - TotalTime: 94057273195 Instance aa4508fe4d6a8e3f:91477db763e7b4af (host=a2124.halxg.cloudera.com:22001) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.9454545454545453 - InactiveTotalTime: 0 - PeakMemoryUsage: 25645056 - PerHostPeakMemUsage: 1850837872 - PrepareTime: 1102224551 - RowsProduced: 1349046 - TotalCpuTime: 161023522715 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 6759838 - TotalStorageWaitTime: 151004727634 - TotalTime: 82528963991 CodeGen - AsyncTotalTime: 0 - CodegenTime: 3988929 - CompileTime: 13071016 - InactiveTotalTime: 0 - LoadTime: 1255361 - ModuleFileSize: 1955380 - OptimizationTime: 202905792 - PrepareTime: 249795264 - TotalTime: 971959748 DataStreamSender (dst_id=18) - AsyncTotalTime: 0 - BytesSent: 56942944 - InactiveTotalTime: 0 - NetworkThroughput(*): 29405050 - OverallThroughput: 16823417 - PeakMemoryUsage: 86016 - SerializeBatchTime: 1597556656 - ThriftTransmitTime(*): 1936502137 - TotalTime: 3384742895 - UncompressedRowBatchSize: 107942164 HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:83.33% 1:16.67% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16666666666666666 - AverageScannerThreadConcurrency: 1.9814814814814814 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268437504 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268437504 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25485312 - PerReadThreadRawHdfsThroughput: 18828162 - RemoteScanRanges: 0 - RowsRead: 2133979 - RowsReturned: 1349046 - RowsReturnedRate: 16992 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 491 - ScannerThreadsTotalWallClockTime: 161023528710 - DelimiterParseTime: 3116466741 - MaterializeTupleTime(*): 381277543 - ScannerThreadsSysTime: 3999000 - ScannerThreadsUserTime: 3523463000 - ScannerThreadsVoluntaryContextSwitches: 41 - TotalRawHdfsReadTime(*): 14257233147 - TotalReadThroughput: 3273615 - TotalTime: 79392151380 Instance aa4508fe4d6a8e3f:91477db763e7b4ad (host=a2124.halxg.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:2/206.94 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.820143884892086 - InactiveTotalTime: 0 - PeakMemoryUsage: 25645056 - PerHostPeakMemUsage: 1815845616 - PrepareTime: 1112967943 - RowsProduced: 1089843 - TotalCpuTime: 127133961182 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 7021587 - TotalStorageWaitTime: 118472831946 - TotalTime: 69538804879 CodeGen - AsyncTotalTime: 0 - CodegenTime: 5810955 - CompileTime: 8785332 - InactiveTotalTime: 0 - LoadTime: 1608672 - ModuleFileSize: 1955380 - OptimizationTime: 273363887 - PrepareTime: 288987501 - TotalTime: 1046684790 DataStreamSender (dst_id=18) - AsyncTotalTime: 0 - BytesSent: 44362706 - InactiveTotalTime: 0 - NetworkThroughput(*): 27146696 - OverallThroughput: 17115149 - PeakMemoryUsage: 86016 - SerializeBatchTime: 1229172712 - ThriftTransmitTime(*): 1634184328 - TotalTime: 2592013985 - UncompressedRowBatchSize: 87202376 HDFS_SCAN_NODE (id=5) Hdfs split stats (:<# splits>/): 0:2/206.94 MB Hdfs Read Thread Concurrency Bucket: 0:83.82% 1:16.18% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16176470588235295 - AverageScannerThreadConcurrency: 1.8602941176470589 - BytesRead: 216992184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 216992184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 216992184 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25485312 - PerReadThreadRawHdfsThroughput: 18045510 - RemoteScanRanges: 0 - RowsRead: 1724693 - RowsReturned: 1089843 - RowsReturnedRate: 16218 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 447 - ScannerThreadsTotalWallClockTime: 127133970045 - DelimiterParseTime: 2587936099 - MaterializeTupleTime(*): 316327430 - ScannerThreadsSysTime: 5998000 - ScannerThreadsUserTime: 2918555000 - ScannerThreadsVoluntaryContextSwitches: 33 - TotalRawHdfsReadTime(*): 12024718809 - TotalReadThroughput: 3122174 - TotalTime: 67196486584 Fragment F04 Instance aa4508fe4d6a8e3f:91477db763e7b4b1 (host=a2124.halxg.cloudera.com:22002) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.9740932642487046 - InactiveTotalTime: 0 - PeakMemoryUsage: 25346000 - PerHostPeakMemUsage: 1850307888 - PrepareTime: 318099329 - RowsProduced: 2142543 - TotalCpuTime: 190768308480 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 3506074 - TotalStorageWaitTime: 180917670370 - TotalTime: 96345920705 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2253043 - CompileTime: 4807644 - InactiveTotalTime: 0 - LoadTime: 1375216 - ModuleFileSize: 1955380 - OptimizationTime: 171601388 - PrepareTime: 311373896 - TotalTime: 492280663 DataStreamSender (dst_id=17) - AsyncTotalTime: 0 - BytesSent: 33140629 - InactiveTotalTime: 0 - NetworkThroughput(*): 29741939 - OverallThroughput: 10494587 - PeakMemoryUsage: 114576 - SerializeBatchTime: 1271167524 - ThriftTransmitTime(*): 1114272622 - TotalTime: 3157878126 - UncompressedRowBatchSize: 60003776 HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:83.85% 1:16.15% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.16145833333333334 - AverageScannerThreadConcurrency: 1.984375 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268437504 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268437504 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25288704 - PerReadThreadRawHdfsThroughput: 15606277 - RemoteScanRanges: 0 - RowsRead: 2142543 - RowsReturned: 2142543 - RowsReturnedRate: 22993 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 608 - ScannerThreadsTotalWallClockTime: 190768313639 - DelimiterParseTime: 3166809833 - MaterializeTupleTime(*): 335827474 - ScannerThreadsSysTime: 2998000 - ScannerThreadsUserTime: 3516465000 - ScannerThreadsVoluntaryContextSwitches: 46 - TotalRawHdfsReadTime(*): 17200611215 - TotalReadThroughput: 2781667 - TotalTime: 93180011676 Instance aa4508fe4d6a8e3f:91477db763e7b4b2 (host=a2124.halxg.cloudera.com:22001) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.9695121951219514 - InactiveTotalTime: 0 - PeakMemoryUsage: 25389648 - PerHostPeakMemUsage: 1850837872 - PrepareTime: 254849384 - RowsProduced: 2133979 - TotalCpuTime: 161500119945 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 4882800 - TotalStorageWaitTime: 151752375508 - TotalTime: 81835182982 CodeGen - AsyncTotalTime: 0 - CodegenTime: 3187808 - CompileTime: 4503459 - InactiveTotalTime: 0 - LoadTime: 1280300 - ModuleFileSize: 1955380 - OptimizationTime: 242891655 - PrepareTime: 237143718 - TotalTime: 499224709 DataStreamSender (dst_id=17) - AsyncTotalTime: 0 - BytesSent: 33317057 - InactiveTotalTime: 0 - NetworkThroughput(*): 30139077 - OverallThroughput: 11302101 - PeakMemoryUsage: 114576 - SerializeBatchTime: 1151015573 - ThriftTransmitTime(*): 1105443838 - TotalTime: 2947864025 - UncompressedRowBatchSize: 59763936 HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:76.07% 1:23.93% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.2392638036809816 - AverageScannerThreadConcurrency: 1.98159509202454 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268437504 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268437504 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25288704 - PerReadThreadRawHdfsThroughput: 15738150 - RemoteScanRanges: 0 - RowsRead: 2133979 - RowsReturned: 2133979 - RowsReturnedRate: 27053 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 554 - ScannerThreadsTotalWallClockTime: 161500124619 - DelimiterParseTime: 3054806079 - MaterializeTupleTime(*): 337381115 - ScannerThreadsSysTime: 9997000 - ScannerThreadsUserTime: 3370487000 - ScannerThreadsVoluntaryContextSwitches: 40 - TotalRawHdfsReadTime(*): 17056483359 - TotalReadThroughput: 3273615 - TotalTime: 78879300687 Instance aa4508fe4d6a8e3f:91477db763e7b4b0 (host=a2124.halxg.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:2/206.94 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.8345323741007196 - InactiveTotalTime: 0 - PeakMemoryUsage: 25367824 - PerHostPeakMemUsage: 1815845616 - PrepareTime: 257545720 - RowsProduced: 1724693 - TotalCpuTime: 127557165721 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 2962472 - TotalStorageWaitTime: 118629687141 - TotalTime: 69247805118 CodeGen - AsyncTotalTime: 0 - CodegenTime: 3128153 - CompileTime: 4829933 - InactiveTotalTime: 0 - LoadTime: 1350344 - ModuleFileSize: 1955380 - OptimizationTime: 196468941 - PrepareTime: 236004482 - TotalTime: 455913600 DataStreamSender (dst_id=17) - AsyncTotalTime: 0 - BytesSent: 26109676 - InactiveTotalTime: 0 - NetworkThroughput(*): 29260584 - OverallThroughput: 11653747 - PeakMemoryUsage: 114576 - SerializeBatchTime: 858617307 - ThriftTransmitTime(*): 892315593 - TotalTime: 2240453258 - UncompressedRowBatchSize: 48301528 HDFS_SCAN_NODE (id=4) Hdfs split stats (:<# splits>/): 0:2/206.94 MB Hdfs Read Thread Concurrency Bucket: 0:81.16% 1:18.84% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.18840579710144928 - AverageScannerThreadConcurrency: 1.8478260869565217 - BytesRead: 216992184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 216992184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 216992184 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 25288704 - PerReadThreadRawHdfsThroughput: 15260016 - RemoteScanRanges: 0 - RowsRead: 1724693 - RowsReturned: 1724693 - RowsReturnedRate: 25741 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 302 - ScannerThreadsTotalWallClockTime: 127557171944 - DelimiterParseTime: 2305864821 - MaterializeTupleTime(*): 251929007 - ScannerThreadsSysTime: 9998000 - ScannerThreadsUserTime: 2571608000 - ScannerThreadsVoluntaryContextSwitches: 33 - TotalRawHdfsReadTime(*): 14219655583 - TotalReadThroughput: 3122174 - TotalTime: 67000697720 Fragment F00 Instance aa4508fe4d6a8e3f:91477db763e7b4b4 (host=a2124.halxg.cloudera.com:22002) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 3.511737089201878 - InactiveTotalTime: 0 - PeakMemoryUsage: 686215184 - PerHostPeakMemUsage: 2055714264 - PrepareTime: 358873644 - RowsProduced: 27087 - TotalCpuTime: 267614851254 - TotalNetworkReceiveTime: 71678758154 - TotalNetworkSendTime: 3446505 - TotalStorageWaitTime: 107977729012 - TotalTime: 106059495339 CodeGen - AsyncTotalTime: 0 - CodegenTime: 5376115 - CompileTime: 36839846 - InactiveTotalTime: 0 - LoadTime: 1247214 - ModuleFileSize: 1955380 - OptimizationTime: 523323501 - PrepareTime: 307392712 - TotalTime: 871755233 DataStreamSender (dst_id=16) - AsyncTotalTime: 0 - BytesSent: 2808377 - InactiveTotalTime: 0 - NetworkThroughput(*): 23725837 - OverallThroughput: 14334924 - PeakMemoryUsage: 151552 - SerializeBatchTime: 96026288 - ThriftTransmitTime(*): 118367875 - TotalTime: 195911531 - UncompressedRowBatchSize: 5719213 HASH_JOIN_NODE (id=8) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 7104987537 - BuildPartitionTime: 14529 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 10693 - GetNewBlockTime: 1357483 - HashBuckets: 2 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 8912912 - PinTime: 0 - ProbeRows: 652595 - ProbeRowsPartitioned: 0 - ProbeTime: 43746034 - RowsReturned: 27087 - RowsReturnedRate: 255 - SpilledPartitions: 0 - TotalTime: 105897235263 - UnpinTime: 13727 EXCHANGE_NODE (id=15) - AsyncTotalTime: 0 - BytesReceived: 44 - ConvertRowBatchTime: 2634 - DeserializeRowBatchTimer: 17069 - FirstBatchArrivalWaitTime: 7102293385 - InactiveTotalTime: 7102514279 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 7102546528 HASH_JOIN_NODE (id=7) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 57542178765 - BuildPartitionTime: 340254649 - BuildRows: 729413 - BuildRowsPartitioned: 729413 - BuildTime: 348533102 - GetNewBlockTime: 30000817 - HashBuckets: 2097152 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 593526784 - PinTime: 0 - ProbeRows: 1354407 - ProbeRowsPartitioned: 0 - ProbeTime: 253009516 - RowsReturned: 652595 - RowsReturnedRate: 6185 - SpilledPartitions: 0 - TotalTime: 105501649459 - UnpinTime: 18415 EXCHANGE_NODE (id=14) - AsyncTotalTime: 0 - BytesReceived: 13357177 - ConvertRowBatchTime: 166835624 - DeserializeRowBatchTimer: 328625475 - FirstBatchArrivalWaitTime: 4479981445 - InactiveTotalTime: 56499874234 - PeakMemoryUsage: 0 - RowsReturned: 729413 - RowsReturnedRate: 12867 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 56684806668 HASH_JOIN_NODE (id=6) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 8081860664 - BuildPartitionTime: 4434240 - BuildRows: 10000 - BuildRowsPartitioned: 10000 - BuildTime: 5197061 - GetNewBlockTime: 325423 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 13033472 - PinTime: 0 - ProbeRows: 1354407 - ProbeRowsPartitioned: 0 - ProbeTime: 358285718 - RowsReturned: 1354407 - RowsReturnedRate: 21360 - SpilledPartitions: 0 - TotalTime: 63407407758 - UnpinTime: 13469 EXCHANGE_NODE (id=13) - AsyncTotalTime: 0 - BytesReceived: 186983 - ConvertRowBatchTime: 2207045 - DeserializeRowBatchTimer: 4667003 - FirstBatchArrivalWaitTime: 5632645538 - InactiveTotalTime: 8064110657 - PeakMemoryUsage: 0 - RowsReturned: 10000 - RowsReturnedRate: 1239 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 8068065960 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:79.62% 1:20.38% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.2037914691943128 - AverageScannerThreadConcurrency: 1.8436018957345972 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268436480 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268436480 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 87957504 - PerReadThreadRawHdfsThroughput: 13343901 - RemoteScanRanges: 0 - RowsRead: 2142543 - RowsReturned: 1354407 - RowsReturnedRate: 24655 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 698 - ScannerThreadsTotalWallClockTime: 194885822947 - DelimiterParseTime: 3359710325 - MaterializeTupleTime(*): 426546761 - ScannerThreadsSysTime: 6998000 - ScannerThreadsUserTime: 4771274000 - ScannerThreadsVoluntaryContextSwitches: 36 - TotalRawHdfsReadTime(*): 20116868537 - TotalReadThroughput: 2532371 - TotalTime: 54934113197 Instance aa4508fe4d6a8e3f:91477db763e7b4b5 (host=a2124.halxg.cloudera.com:22001) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 3.6 - InactiveTotalTime: 0 - PeakMemoryUsage: 686329872 - PerHostPeakMemUsage: 1850837872 - PrepareTime: 349661022 - RowsProduced: 27140 - TotalCpuTime: 240548080411 - TotalNetworkReceiveTime: 72181253528 - TotalNetworkSendTime: 1058952 - TotalStorageWaitTime: 74161554351 - TotalTime: 92470648656 CodeGen - AsyncTotalTime: 0 - CodegenTime: 4931450 - CompileTime: 36981996 - InactiveTotalTime: 0 - LoadTime: 1368842 - ModuleFileSize: 1955380 - OptimizationTime: 364391525 - PrepareTime: 306340429 - TotalTime: 711959863 DataStreamSender (dst_id=16) - AsyncTotalTime: 0 - BytesSent: 2813106 - InactiveTotalTime: 0 - NetworkThroughput(*): 24562963 - OverallThroughput: 12767071 - PeakMemoryUsage: 151552 - SerializeBatchTime: 108596888 - ThriftTransmitTime(*): 114526329 - TotalTime: 220340746 - UncompressedRowBatchSize: 5730412 HASH_JOIN_NODE (id=8) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 7272620035 - BuildPartitionTime: 18707 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 7384 - GetNewBlockTime: 1135981 - HashBuckets: 2 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 9003024 - PinTime: 0 - ProbeRows: 650495 - ProbeRowsPartitioned: 0 - ProbeTime: 53655441 - RowsReturned: 27140 - RowsReturnedRate: 294 - SpilledPartitions: 0 - TotalTime: 92275356943 - UnpinTime: 18435 EXCHANGE_NODE (id=15) - AsyncTotalTime: 0 - BytesReceived: 44 - ConvertRowBatchTime: 2614 - DeserializeRowBatchTimer: 22710 - FirstBatchArrivalWaitTime: 7268839567 - InactiveTotalTime: 7270137120 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 7270170953 HASH_JOIN_NODE (id=7) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 57772508771 - BuildPartitionTime: 336528856 - BuildRows: 729413 - BuildRowsPartitioned: 729413 - BuildTime: 397213343 - GetNewBlockTime: 29173024 - HashBuckets: 2097152 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 593526784 - PinTime: 0 - ProbeRows: 1349046 - ProbeRowsPartitioned: 0 - ProbeTime: 258556066 - RowsReturned: 650495 - RowsReturnedRate: 7080 - SpilledPartitions: 0 - TotalTime: 91867800621 - UnpinTime: 16844 EXCHANGE_NODE (id=14) - AsyncTotalTime: 0 - BytesReceived: 13357177 - ConvertRowBatchTime: 167846648 - DeserializeRowBatchTimer: 325907385 - FirstBatchArrivalWaitTime: 4646668762 - InactiveTotalTime: 56668608947 - PeakMemoryUsage: 0 - RowsReturned: 729413 - RowsReturnedRate: 12829 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 56854869315 HASH_JOIN_NODE (id=6) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 8247711085 - BuildPartitionTime: 4571419 - BuildRows: 10000 - BuildRowsPartitioned: 10000 - BuildTime: 4932836 - GetNewBlockTime: 574155 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 13033472 - PinTime: 0 - ProbeRows: 1349046 - ProbeRowsPartitioned: 0 - ProbeTime: 382543983 - RowsReturned: 1349046 - RowsReturnedRate: 25826 - SpilledPartitions: 0 - TotalTime: 52235734826 - UnpinTime: 19550 EXCHANGE_NODE (id=13) - AsyncTotalTime: 0 - BytesReceived: 186983 - ConvertRowBatchTime: 1977090 - DeserializeRowBatchTimer: 5017912 - FirstBatchArrivalWaitTime: 5798931767 - InactiveTotalTime: 8230621915 - PeakMemoryUsage: 0 - RowsReturned: 10000 - RowsReturnedRate: 1214 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 8233740663 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:84.78% 1:15.22% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.15217391304347827 - AverageScannerThreadConcurrency: 1.809782608695652 - BytesRead: 268437504 - BytesReadDataNodeCache: 0 - BytesReadLocal: 268436480 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 268436480 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 88072192 - PerReadThreadRawHdfsThroughput: 17842568 - RemoteScanRanges: 0 - RowsRead: 2133979 - RowsReturned: 1349046 - RowsReturnedRate: 30963 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 718 - ScannerThreadsTotalWallClockTime: 167255237999 - DelimiterParseTime: 3461340581 - MaterializeTupleTime(*): 437502433 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 4875258000 - ScannerThreadsVoluntaryContextSwitches: 34 - TotalRawHdfsReadTime(*): 15044779105 - TotalReadThroughput: 2902016 - TotalTime: 43568541082 Instance aa4508fe4d6a8e3f:91477db763e7b4b3 (host=a2124.halxg.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:2/206.94 MB - AsyncTotalTime: 0 - AverageThreadTokens: 2.8922155688622753 - InactiveTotalTime: 0 - PeakMemoryUsage: 652705808 - PerHostPeakMemUsage: 1815845616 - PrepareTime: 321172696 - RowsProduced: 21644 - TotalCpuTime: 157667615998 - TotalNetworkReceiveTime: 71858942567 - TotalNetworkSendTime: 1418910 - TotalStorageWaitTime: 46619025009 - TotalTime: 83379285649 CodeGen - AsyncTotalTime: 0 - CodegenTime: 4131234 - CompileTime: 53588821 - InactiveTotalTime: 0 - LoadTime: 1403233 - ModuleFileSize: 1955380 - OptimizationTime: 480776258 - PrepareTime: 275986656 - TotalTime: 814473371 DataStreamSender (dst_id=16) - AsyncTotalTime: 0 - BytesSent: 2247013 - InactiveTotalTime: 0 - NetworkThroughput(*): 24267492 - OverallThroughput: 14485427 - PeakMemoryUsage: 151552 - SerializeBatchTime: 76102376 - ThriftTransmitTime(*): 92593540 - TotalTime: 155122316 - UncompressedRowBatchSize: 4569972 HASH_JOIN_NODE (id=8) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 7168373763 - BuildPartitionTime: 12973 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 12825 - GetNewBlockTime: 578779 - HashBuckets: 2 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 9003024 - PinTime: 0 - ProbeRows: 525821 - ProbeRowsPartitioned: 0 - ProbeTime: 36395409 - RowsReturned: 21644 - RowsReturnedRate: 259 - SpilledPartitions: 0 - TotalTime: 83252659421 - UnpinTime: 20290 EXCHANGE_NODE (id=15) - AsyncTotalTime: 0 - BytesReceived: 44 - ConvertRowBatchTime: 3429 - DeserializeRowBatchTimer: 23623 - FirstBatchArrivalWaitTime: 7165052635 - InactiveTotalTime: 7166367626 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 7166409463 HASH_JOIN_NODE (id=7) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously - AsyncTotalTime: 57630575146 - BuildPartitionTime: 346318063 - BuildRows: 729413 - BuildRowsPartitioned: 729413 - BuildTime: 368374370 - GetNewBlockTime: 31743973 - HashBuckets: 2097152 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 593526784 - PinTime: 0 - ProbeRows: 1089843 - ProbeRowsPartitioned: 0 - ProbeTime: 189252470 - RowsReturned: 525821 - RowsReturnedRate: 6343 - SpilledPartitions: 0 - TotalTime: 82896640198 - UnpinTime: 25338 EXCHANGE_NODE (id=14) - AsyncTotalTime: 0 - BytesReceived: 13357177 - ConvertRowBatchTime: 168529263 - DeserializeRowBatchTimer: 342957418 - FirstBatchArrivalWaitTime: 4542910306 - InactiveTotalTime: 56550428872 - PeakMemoryUsage: 0 - RowsReturned: 729413 - RowsReturnedRate: 12855 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 56739293826 HASH_JOIN_NODE (id=6) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - AsyncTotalTime: 0 - BuildPartitionTime: 4676069 - BuildRows: 10000 - BuildRowsPartitioned: 10000 - BuildTime: 9556920 - GetNewBlockTime: 574396 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 3 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 32 - PeakMemoryUsage: 13033472 - PinTime: 0 - ProbeRows: 1089843 - ProbeRowsPartitioned: 0 - ProbeTime: 301892472 - RowsReturned: 1089843 - RowsReturnedRate: 27380 - SpilledPartitions: 0 - TotalTime: 39803234819 - UnpinTime: 19013 EXCHANGE_NODE (id=13) - AsyncTotalTime: 0 - BytesReceived: 186983 - ConvertRowBatchTime: 3397174 - DeserializeRowBatchTimer: 4512146 - FirstBatchArrivalWaitTime: 5695303144 - InactiveTotalTime: 8127599765 - PeakMemoryUsage: 0 - RowsReturned: 10000 - RowsReturnedRate: 1229 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 8132764267 HDFS_SCAN_NODE (id=1) Hdfs split stats (:<# splits>/): 0:2/206.94 MB Hdfs Read Thread Concurrency Bucket: 0:81.88% 1:18.12% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:2 ExecOption: Codegen enabled: 2 out of 2 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.18120805369127516 - AverageScannerThreadConcurrency: 1.2416107382550337 - BytesRead: 216992184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 216992184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 216992184 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 2 - PeakMemoryUsage: 54448128 - PerReadThreadRawHdfsThroughput: 17139186 - RemoteScanRanges: 0 - RowsRead: 1724693 - RowsReturned: 1089843 - RowsReturnedRate: 34801 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 438 - ScannerThreadsTotalWallClockTime: 92868666360 - DelimiterParseTime: 2586862414 - MaterializeTupleTime(*): 325451392 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 2924554000 - ScannerThreadsVoluntaryContextSwitches: 30 - TotalRawHdfsReadTime(*): 12660588662 - TotalReadThroughput: 2511571 - TotalTime: 31315578781 Fragment F03 Instance aa4508fe4d6a8e3f:91477db763e7b4b6 (host=a2124.halxg.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:1/2.15 KB - AsyncTotalTime: 0 - AverageThreadTokens: 1.875 - InactiveTotalTime: 0 - PeakMemoryUsage: 66544 - PerHostPeakMemUsage: 120999472 - PrepareTime: 271264033 - RowsProduced: 1 - TotalCpuTime: 9403777724 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 2001406 - TotalStorageWaitTime: 5399470499 - TotalTime: 7416248118 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2829575 - CompileTime: 6439215 - InactiveTotalTime: 0 - LoadTime: 1370031 - ModuleFileSize: 1955380 - OptimizationTime: 240108859 - PrepareTime: 224298760 - TotalTime: 473619911 DataStreamSender (dst_id=15) - AsyncTotalTime: 0 - BytesSent: 144 - InactiveTotalTime: 0 - NetworkThroughput(*): 41818 - OverallThroughput: 67209 - PeakMemoryUsage: 16368 - SerializeBatchTime: 23044 - ThriftTransmitTime(*): 3443482 - TotalTime: 2142565 - UncompressedRowBatchSize: 144 HDFS_SCAN_NODE (id=3) Hdfs split stats (:<# splits>/): 0:1/2.15 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 ExecOption: Codegen enabled: 1 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 2199 - BytesReadDataNodeCache: 0 - BytesReadLocal: 2199 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 2199 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 41984 - PerReadThreadRawHdfsThroughput: 967967 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 7140281624 - DelimiterParseTime: 20059 - MaterializeTupleTime(*): 4900 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 4 - TotalRawHdfsReadTime(*): 2271771 - TotalReadThroughput: 293 - TotalTime: 7453715653 Fragment F02 Instance aa4508fe4d6a8e3f:91477db763e7b4b7 (host=a2124.halxg.cloudera.com:22002) Hdfs split stats (:<# splits>/): 0:1/128.00 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9912280701754386 - InactiveTotalTime: 0 - PeakMemoryUsage: 16949248 - PerHostPeakMemUsage: 1838780688 - PrepareTime: 255322579 - RowsProduced: 574226 - TotalCpuTime: 60536152457 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1313620 - TotalStorageWaitTime: 53426926628 - TotalTime: 56901435021 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2953237 - CompileTime: 4422507 - InactiveTotalTime: 0 - LoadTime: 1271611 - ModuleFileSize: 1955380 - OptimizationTime: 399184082 - PrepareTime: 249755381 - TotalTime: 655937342 DataStreamSender (dst_id=14) - AsyncTotalTime: 0 - BytesSent: 32690634 - InactiveTotalTime: 0 - NetworkThroughput(*): 24409168 - OverallThroughput: 70920511 - PeakMemoryUsage: 12288 - SerializeBatchTime: 444898356 - ThriftTransmitTime(*): 1339276837 - TotalTime: 460947522 - UncompressedRowBatchSize: 63745866 HDFS_SCAN_NODE (id=2) Hdfs split stats (:<# splits>/): 0:1/128.00 MB Hdfs Read Thread Concurrency Bucket: 0:84.07% 1:15.93% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 ExecOption: Codegen enabled: 1 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.1592920353982301 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 134218752 - BytesReadDataNodeCache: 0 - BytesReadLocal: 134218752 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 134218752 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 16928768 - PerReadThreadRawHdfsThroughput: 17701830 - RemoteScanRanges: 0 - RowsRead: 1181503 - RowsReturned: 574226 - RowsReturnedRate: 10174 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 166 - ScannerThreadsTotalWallClockTime: 56650426351 - DelimiterParseTime: 1087180660 - MaterializeTupleTime(*): 103771112 - ScannerThreadsSysTime: 1999000 - ScannerThreadsUserTime: 1206816000 - ScannerThreadsVoluntaryContextSwitches: 19 - TotalRawHdfsReadTime(*): 7582196134 - TotalReadThroughput: 2354696 - TotalTime: 56436695655 Instance aa4508fe4d6a8e3f:91477db763e7b4b8 (host=a2124.halxg.cloudera.com:22001) Hdfs split stats (:<# splits>/): 0:1/34.56 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.975609756097561 - InactiveTotalTime: 0 - PeakMemoryUsage: 16883712 - PerHostPeakMemUsage: 693907728 - PrepareTime: 312206383 - RowsProduced: 155187 - TotalCpuTime: 22612206754 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 1257615 - TotalStorageWaitTime: 17970153120 - TotalTime: 20274476243 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2954740 - CompileTime: 4435919 - InactiveTotalTime: 0 - LoadTime: 1415480 - ModuleFileSize: 1955380 - OptimizationTime: 341633639 - PrepareTime: 305002628 - TotalTime: 655185277 DataStreamSender (dst_id=14) - AsyncTotalTime: 0 - BytesSent: 7380921 - InactiveTotalTime: 0 - NetworkThroughput(*): 19457403 - OverallThroughput: 58763105 - PeakMemoryUsage: 12288 - SerializeBatchTime: 121203905 - ThriftTransmitTime(*): 379337402 - TotalTime: 125604678 - UncompressedRowBatchSize: 17227605 HDFS_SCAN_NODE (id=2) Hdfs split stats (:<# splits>/): 0:1/34.56 MB Hdfs Read Thread Concurrency Bucket: 0:82.5% 1:17.5% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 ExecOption: Codegen enabled: 1 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.175 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 36234433 - BytesReadDataNodeCache: 0 - BytesReadLocal: 36234433 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 36234433 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 16863232 - PerReadThreadRawHdfsThroughput: 14750152 - RemoteScanRanges: 0 - RowsRead: 318497 - RowsReturned: 155187 - RowsReturnedRate: 7702 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 40 - ScannerThreadsTotalWallClockTime: 19959508827 - DelimiterParseTime: 322344405 - MaterializeTupleTime(*): 31706526 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 358945000 - ScannerThreadsVoluntaryContextSwitches: 9 - TotalRawHdfsReadTime(*): 2456546283 - TotalReadThroughput: 1767533 - TotalTime: 20146569528 Fragment F01 Instance aa4508fe4d6a8e3f:91477db763e7b4b9 (host=a2124.halxg.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:1/1.33 MB - AsyncTotalTime: 0 - AverageThreadTokens: 1.9375 - InactiveTotalTime: 0 - PeakMemoryUsage: 2244608 - PerHostPeakMemUsage: 125284944 - PrepareTime: 260327422 - RowsProduced: 10000 - TotalCpuTime: 10493117860 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 2674013 - TotalStorageWaitTime: 5136317846 - TotalTime: 7851134868 CodeGen - AsyncTotalTime: 0 - CodegenTime: 2444843 - CompileTime: 6238460 - InactiveTotalTime: 0 - LoadTime: 1469385 - ModuleFileSize: 1955380 - OptimizationTime: 183247851 - PrepareTime: 253613192 - TotalTime: 447407801 DataStreamSender (dst_id=13) - AsyncTotalTime: 0 - BytesSent: 560961 - InactiveTotalTime: 0 - NetworkThroughput(*): 22574216 - OverallThroughput: 46480024 - PeakMemoryUsage: 12288 - SerializeBatchTime: 9240353 - ThriftTransmitTime(*): 24849633 - TotalTime: 12068862 - UncompressedRowBatchSize: 1620132 HDFS_SCAN_NODE (id=0) Hdfs split stats (:<# splits>/): 0:1/1.33 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% File Formats: TEXT/NONE:1 ExecOption: Codegen enabled: 1 out of 1 - AsyncTotalTime: 0 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 1399184 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1399184 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1399184 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumDisksAccessed: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 2183168 - PerReadThreadRawHdfsThroughput: 73743009 - RemoteScanRanges: 0 - RowsRead: 10000 - RowsReturned: 10000 - RowsReturnedRate: 1275 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 1 - ScannerThreadsTotalWallClockTime: 7587448862 - DelimiterParseTime: 10012517 - MaterializeTupleTime(*): 1756773 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 11998000 - ScannerThreadsVoluntaryContextSwitches: 7 - TotalRawHdfsReadTime(*): 18973785 - TotalReadThroughput: 174898 - TotalTime: 7837430502