Query (id=4d46f4c42facb2e7:50aa3b04b8bd0e85) Summary Session ID: 2641121c428ce472:1bfd7069a17d9594 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2016-02-23 14:16:57.749818000 End Time: 2016-02-23 14:37:35.918996000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.5.0-cdh5.7.0 DEBUG (build 22f71f3e82151c1064bf2983dd3e9025c4cce5b3) User: jenkins Connected User: jenkins Delegated User: Network Address: 172.26.26.3:56011 Default Db: tpch_10_nested_parquet Sql Statement: /* Mem: 514 MB. Coordinator: impala-stress-cdh570-2.vpc.cloudera.com. */ select s_name, count(*) as numwait from supplier s, customer c, c.c_orders o, o.o_lineitems l1, region.r_nations n where s_suppkey = l1.l_suppkey and o_orderstatus = 'F' and l1.l_receiptdate > l1.l_commitdate and exists ( select * from o.o_lineitems l2 where l2.l_suppkey <> l1.l_suppkey ) and not exists ( select * from o.o_lineitems l3 where l3.l_suppkey <> l1.l_suppkey and l3.l_receiptdate > l3.l_commitdate ) and s_nationkey = n_nationkey and n_name = 'SAUDI ARABIA' group by s_name order by numwait desc, s_name limit 100 Coordinator: impala-stress-cdh570-2.vpc.cloudera.com:22000 Query Options (non default): ABORT_ON_ERROR=1,MEM_LIMIT=538968064,RUNTIME_FILTER_MODE=0,DISABLE_ROW_RUNTIME_FILTERING=1 Plan: ---------------- Estimated Per-Host Requirements: Memory=300.62MB VCores=2 WARNING: The following tables are missing relevant table and/or column statistics. tpch_10_nested_parquet.customer, tpch_10_nested_parquet.region 25:MERGING-EXCHANGE [UNPARTITIONED] | order by: count(*) DESC, s_name ASC | limit: 100 | hosts=4 per-host-mem=unavailable | tuple-ids=10 row-size=42B cardinality=100 | 20:TOP-N [LIMIT=100] | order by: count(*) DESC, s_name ASC | hosts=4 per-host-mem=4.10KB | tuple-ids=10 row-size=42B cardinality=100 | 24:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: s_name | hosts=4 per-host-mem=10.00MB | tuple-ids=9 row-size=42B cardinality=96988 | 23:EXCHANGE [HASH(s_name)] | hosts=4 per-host-mem=0B | tuple-ids=9 row-size=42B cardinality=96988 | 19:AGGREGATE [STREAMING] | output: count(*) | group by: s_name | hosts=4 per-host-mem=10.00MB | tuple-ids=9 row-size=42B cardinality=96988 | 18:SUBPLAN | hosts=4 per-host-mem=0B | tuple-ids=3,2,1,0,4 row-size=182B cardinality=150000000 | |--16:NESTED LOOP JOIN [RIGHT ANTI JOIN] | | join predicates: l3.l_suppkey != l1.l_suppkey | | hosts=4 per-host-mem=182B | | tuple-ids=3,2,1,0,4 row-size=182B cardinality=1 | | | |--14:SINGULAR ROW SRC | | parent-subplan=18 | | hosts=4 per-host-mem=0B | | tuple-ids=3,2,1,0,4 row-size=182B cardinality=1 | | | 15:UNNEST [o.o_lineitems l3] | parent-subplan=18 | hosts=4 per-host-mem=0B | tuple-ids=7 row-size=40B cardinality=10 | 17:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: s_nationkey = n_nationkey | hosts=4 per-host-mem=100B | tuple-ids=3,2,1,0,4 row-size=182B cardinality=150000000 | |--22:EXCHANGE [BROADCAST] | | hosts=1 per-host-mem=0B | | tuple-ids=4 row-size=18B cardinality=5 | | | 12:SCAN HDFS [tpch_10_nested_parquet.region.r_nations n, RANDOM] | partitions=1/1 files=1 size=4.17KB | predicates: n_name = 'SAUDI ARABIA' | table stats: 5 rows total | column stats: all | hosts=1 per-host-mem=32.00MB | tuple-ids=4 row-size=18B cardinality=5 | 13:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: l1.l_suppkey = s_suppkey | hosts=4 per-host-mem=4.62MB | tuple-ids=3,2,1,0 row-size=164B cardinality=150000000 | |--21:EXCHANGE [BROADCAST] | | hosts=4 per-host-mem=0B | | tuple-ids=0 row-size=44B cardinality=100000 | | | 00:SCAN HDFS [tpch_10_nested_parquet.supplier s, RANDOM] | partitions=1/1 files=5 size=1.09GB | table stats: 100000 rows total | column stats: all | hosts=4 per-host-mem=264.00MB | tuple-ids=0 row-size=44B cardinality=100000 | 02:SUBPLAN | hosts=4 per-host-mem=0B | tuple-ids=3,2,1 row-size=120B cardinality=150000000 | |--11:NESTED LOOP JOIN [CROSS JOIN] | | hosts=4 per-host-mem=16B | | tuple-ids=3,2,1 row-size=120B cardinality=100 | | | |--03:SINGULAR ROW SRC | | parent-subplan=02 | | hosts=4 per-host-mem=0B | | tuple-ids=1 row-size=16B cardinality=1 | | | 05:SUBPLAN | | hosts=4 per-host-mem=0B | | tuple-ids=3,2 row-size=104B cardinality=100 | | | |--10:NESTED LOOP JOIN [LEFT SEMI JOIN] | | | join predicates: l2.l_suppkey != l1.l_suppkey | | | hosts=4 per-host-mem=80B | | | tuple-ids=3,2 row-size=104B cardinality=10 | | | | | |--08:UNNEST [o.o_lineitems l2] | | | parent-subplan=05 | | | hosts=4 per-host-mem=0B | | | tuple-ids=5 row-size=8B cardinality=10 | | | | | 09:NESTED LOOP JOIN [CROSS JOIN] | | | hosts=4 per-host-mem=64B | | | tuple-ids=3,2 row-size=104B cardinality=10 | | | | | |--06:SINGULAR ROW SRC | | | parent-subplan=05 | | | hosts=4 per-host-mem=0B | | | tuple-ids=2 row-size=64B cardinality=1 | | | | | 07:UNNEST [o.o_lineitems l1] | | parent-subplan=05 | | hosts=4 per-host-mem=0B | | tuple-ids=3 row-size=0B cardinality=10 | | | 04:UNNEST [c.c_orders o] | parent-subplan=02 | hosts=4 per-host-mem=0B | tuple-ids=2 row-size=0B cardinality=10 | 01:SCAN HDFS [tpch_10_nested_parquet.customer c, RANDOM] partitions=1/1 files=31 size=5.68GB predicates: !empty(c.c_orders) predicates on o: !empty(o.o_lineitems), o_orderstatus = 'F' predicates on l1: l1.l_receiptdate > l1.l_commitdate predicates on l3: l3.l_receiptdate > l3.l_commitdate table stats: 1500000 rows total column stats: unavailable hosts=4 per-host-mem=88.00MB tuple-ids=1 row-size=16B cardinality=1500000 ---------------- Estimated Per-Host Mem: 315218596 Estimated Per-Host VCores: 2 Tables Missing Stats: tpch_10_nested_parquet.customer,tpch_10_nested_parquet.region Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------------------ 25:MERGING-EXCHANGE 1 908.209us 908.209us 400 100 0 -1.00 B UNPARTITIONED 20:TOP-N 4 447.037ms 760.580ms 400 100 20.00 KB 4.10 KB 24:AGGREGATE 4 46s051ms 1m7s 4.01K 96.99K 2.57 MB 10.00 MB FINALIZE 23:EXCHANGE 4 705.405us 842.185us 10.98K 96.99K 0 0 HASH(s_name) 19:AGGREGATE 4 51s560ms 1m22s 10.98K 96.99K 24.78 MB 10.00 MB STREAMING 18:SUBPLAN 4 233.184ms 932.737ms 0 150.00M 23.53 MB 0 |--16:NESTED LOOP JOIN 4 5s515ms 8s446ms 0 1 48.01 KB 182.00 B RIGHT ANTI JOIN | |--14:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | 15:UNNEST 4 1s371ms 1s988ms 0 10 0 0 o.o_lineitems l3 17:HASH JOIN 4 4s551ms 6s226ms 530.33K 150.00M 23.58 MB 100.00 B INNER JOIN, BROADCAST |--22:EXCHANGE 4 25.506us 34.116us 1 5 0 0 BROADCAST | 12:SCAN HDFS 1 18s385ms 18s385ms 1 5 42.00 KB 32.00 MB tpch_10_nested_parquet.regi... 13:HASH JOIN 4 15s185ms 18s115ms 17.56M 150.00M 36.51 MB 4.62 MB INNER JOIN, BROADCAST |--21:EXCHANGE 4 1s418ms 2s441ms 100.00K 100.00K 0 0 BROADCAST | 00:SCAN HDFS 4 1m30s 4m42s 100.00K 100.00K 2.11 MB 264.00 MB tpch_10_nested_parquet.supp... 02:SUBPLAN 4 2s917ms 3s214ms 16.84M 150.00M 23.50 MB 0 |--11:NESTED LOOP JOIN 4 15s131ms 15s794ms 0 100 24.00 KB 16.00 B CROSS JOIN | |--03:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | 05:SUBPLAN 4 25s953ms 26s990ms 0 100 8.00 KB 0 | |--10:NESTED LOOP JOIN 4 2m22s 2m26s 0 10 24.00 KB 80.00 B LEFT SEMI JOIN | | |--08:UNNEST 4 15s238ms 15s346ms 0 10 0 0 o.o_lineitems l2 | | 09:NESTED LOOP JOIN 4 50s835ms 51s980ms 0 10 16.00 KB 64.00 B CROSS JOIN | | |--06:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | | 07:UNNEST 4 14s672ms 15s636ms 0 10 0 0 o.o_lineitems l1 | 04:UNNEST 4 2s675ms 3s043ms 0 10 0 0 c.c_orders o 01:SCAN HDFS 4 1m48s 2m13s 994.24K 1.50M 67.63 MB 88.00 MB tpch_10_nested_parquet.cust... Planner Timeline Analysis finished: 533406571 Equivalence classes computed: 590421322 Single node plan created: 1357585118 Distributed plan created: 5334391880 Lineage info computed: 5335321935 Planning finished: 5607378513 Query Timeline Start execution: 89188 Planning finished: 6229278690 Ready to start 13 remote fragments: 6436168700 All 13 remote fragments started: 442795943556 Rows available: 1238098547724 First row fetched: 1238136760081 Unregister query: 1238169166938 ImpalaServer - ClientFetchWaitTimer: 64489194 - InactiveTotalTime: 0 - RowMaterializationTimer: 133387 - TotalTime: 0 Execution Profile 4d46f4c42facb2e7:50aa3b04b8bd0e85 Fragment start latencies: Count: 13, 25th %-ile: 181ms, 50th %-ile: 270ms, 75th %-ile: 347ms, 90th %-ile: 471ms, 95th %-ile: 471ms, 99.9th %-ile: 479ms Per Node Peak Memory Usage: impala-stress-cdh570-5.vpc.cloudera.com:22000(92.96 MB) impala-stress-cdh570-4.vpc.cloudera.com:22000(70.65 MB) impala-stress-cdh570-3.vpc.cloudera.com:22000(81.46 MB) impala-stress-cdh570-2.vpc.cloudera.com:22000(74.81 MB) - FiltersReceived: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 1231537691290 Coordinator Fragment F04 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 31392 - PerHostPeakMemUsage: 0 - PrepareTime: 128096 - RowsProduced: 100 - TotalCpuTime: 1231705282784 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 794720197896 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4718608 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=25) - BytesReceived: 7314 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 160945 - FirstBatchArrivalWaitTime: 794717729456 - InactiveTotalTime: 794717741928 - MergeGetNext: 793067 - MergeGetNextBatch: 34962 - PeakMemoryUsage: 0 - RowsReturned: 400 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 794718650137 Averaged Fragment F03 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:13m15s max:13m15s mean: 13m15s stddev:50.309ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 4 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737498 - PerHostPeakMemUsage: 83854236 - PrepareTime: 46490140012 - RowsProduced: 100 - TotalCpuTime: 150405554209 - TotalNetworkReceiveTime: 645309024329 - TotalNetworkSendTime: 1311548 - TotalStorageWaitTime: 0 - TotalTime: 692658982245 CodeGen - CodegenTime: 1059991408 - CompileTime: 2050194889 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 100397229866 - PrepareTime: 43242378856 - TotalTime: 146447679745 DataStreamSender (dst_id=25) - BytesSent: 1828 - InactiveTotalTime: 0 - NetworkThroughput(*): 3195497 - OverallThroughput: 7347313 - PeakMemoryUsage: 4096 - SerializeBatchTime: 93174 - ThriftTransmitTime(*): 1323560 - TotalTime: 257749 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 691808630624 AGGREGATION_NODE (id=24) - BuildTime: 2427460 - GetNewBlockTime: 1095830606 - GetResultsTime: 449753 - HTResizeTime: 939412 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2691418 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1002 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 691361593143 - UnpinTime: 23697 EXCHANGE_NODE (id=23) - BytesReceived: 50581 - ConvertRowBatchTime: 537380 - DeserializeRowBatchTimer: 838889 - FirstBatchArrivalWaitTime: 594148835863 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 2745 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 645309626113 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4904288 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 Averaged Fragment F00 split sizes: min: 1.41 GB, max: 1.44 GB, avg: 1.42 GB, stddev: 14.11 MB completion times: min:12m24s max:13m15s mean: 12m47s stddev:18s238ms execution rates: min:1.82 MB/sec max:1.95 MB/sec mean:1.90 MB/sec stddev:50.15 KB/sec num instances: 4 - AverageThreadTokens: 1.475670419820006 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 81476508 - PerHostPeakMemUsage: 83854236 - PrepareTime: 53881306159 - RowsProduced: 2745 - TotalCpuTime: 891097654745 - TotalNetworkReceiveTime: 228785908746 - TotalNetworkSendTime: 4325785 - TotalStorageWaitTime: 3946581800 - TotalTime: 648842279618 CodeGen - CodegenTime: 927913260 - CompileTime: 4144308180 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 113209406319 - PrepareTime: 46679262397 - TotalTime: 165294235987 DataStreamSender (dst_id=23) - BytesSent: 50581 - InactiveTotalTime: 0 - NetworkThroughput(*): 4132514 - OverallThroughput: 8417221 - PeakMemoryUsage: 126976 - SerializeBatchTime: 2277665 - ThriftTransmitTime(*): 9627805 - TotalTime: 4532610 - UncompressedRowBatchSize: 148294 AGGREGATION_NODE (id=19) - GetNewBlockTime: 461602927 - GetResultsTime: 806492 - HTResizeTime: 0 - HashBuckets: 12288 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25457920 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 2745 - RowsReturnedRate: 3 - StreamingTime: 2213921703 - TotalTime: 646286561901 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24147200 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 594725949116 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 5515066446 - PeakMemoryUsage: 38920 - ProbeRows: 479440 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 6886658403 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 2.712086160509737 - InactiveTotalTime: 0 - MaxCollectionSize: 5 - MinCollectionSize: 0 - NumCollections: 132583 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1371591956 HASH_JOIN_NODE (id=17) - BuildPartitionTime: 6959 - BuildRows: 0 - BuildRowsPartitioned: 0 - BuildTime: 5517 - GetNewBlockTime: 184834059 - HashBuckets: 0 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 75 - LocalTime: 4551579214 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24188172 - PinTime: 0 - ProbeRows: 4390984 - ProbeRowsPartitioned: 0 - ProbeTime: 1011056488 - RowsReturned: 132583 - RowsReturnedRate: 220 - SpilledPartitions: 0 - TotalTime: 589193195925 - UnpinTime: 21089 EXCHANGE_NODE (id=22) - BytesReceived: 30 - ConvertRowBatchTime: 3127 - DeserializeRowBatchTimer: 15204 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 27112 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 25506 HASH_JOIN_NODE (id=13) - BuildPartitionTime: 3097620467 - BuildRows: 99488 - BuildRowsPartitioned: 99488 - BuildTime: 703731065 - GetNewBlockTime: 333645 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 15185848503 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37762304 - PinTime: 0 - ProbeRows: 4413590 - ProbeRowsPartitioned: 0 - ProbeTime: 7224045201 - RowsReturned: 4390984 - RowsReturnedRate: 7523 - SpilledPartitions: 0 - TotalTime: 584641591204 - UnpinTime: 16757 EXCHANGE_NODE (id=21) - BytesReceived: 2381578 - ConvertRowBatchTime: 1277315393 - DeserializeRowBatchTimer: 4839460289 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 99488 - RowsReturnedRate: 440 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 230196653025 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24114432 - RowsReturned: 4210688 - RowsReturnedRate: 12443 - TotalTime: 339259089676 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15131757268 - PeakMemoryUsage: 24576 - ProbeRows: 4413590 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 228186415542 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 213054658273 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 4 - BuildTime: 9960651592 - InactiveTotalTime: 0 - LocalTime: 142243866569 - PeakMemoryUsage: 24576 - ProbeRows: 4580350 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 184425471473 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205617238616756 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667768 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15238510112 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 50835144358 - PeakMemoryUsage: 16384 - ProbeRows: 4580350 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 65507585076 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.7463907123699363 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667768 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14672440718 UNNEST_NODE (id=4) - AvgCollectionSize: 6.709651421602359 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248560 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2675188689 HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.0017020337053848997 - AverageScannerThreadConcurrency: 0.9996268656716418 - BytesRead: 258737470 - BytesReadDataNodeCache: 0 - BytesReadLocal: 258507070 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 258507070 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 15 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 57312136 - PerReadThreadRawHdfsThroughput: 473663477 - RemoteScanRanges: 2 - RowsRead: 49114539 - RowsReturned: 248560 - RowsReturnedRate: 2414 - ScanRangesComplete: 15 - ScannerThreadsInvoluntaryContextSwitches: 47105 - ScannerThreadsTotalWallClockTime: 356727287785 - MaterializeTupleTime(*): 350728260703 - ScannerThreadsSysTime: 404438250 - ScannerThreadsUserTime: 102253954750 - ScannerThreadsVoluntaryContextSwitches: 400 - TotalRawHdfsReadTime(*): 593258483 - TotalReadThroughput: 365446 - TotalTime: 108154747370 Averaged Fragment F02 split sizes: min: 4.17 KB, max: 4.17 KB, avg: 4.17 KB, stddev: 0 completion times: min:1m24s max:1m24s mean: 1m24s stddev:0.000ns execution rates: min:50.00 B/sec max:50.00 B/sec mean:50.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.2278481012658229 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 73024 - PerHostPeakMemUsage: 7695240 - PrepareTime: 259507 - RowsProduced: 1 - TotalCpuTime: 25176618896 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 59531281644 - TotalStorageWaitTime: 15099898078 - TotalTime: 20065395070 DataStreamSender (dst_id=22) - BytesSent: 164 - InactiveTotalTime: 0 - NetworkThroughput(*): 1 - OverallThroughput: 4512 - PeakMemoryUsage: 21824 - SerializeBatchTime: 36233182 - ThriftTransmitTime(*): 121445498990 - TotalTime: 36343092 - UncompressedRowBatchSize: 176 HDFS_SCAN_NODE (id=12) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 4758 - BytesReadDataNodeCache: 0 - BytesReadLocal: 4758 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 4758 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 43008 - PerReadThreadRawHdfsThroughput: 146874517 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 18132821884 - MaterializeTupleTime(*): 70045 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 999000 - ScannerThreadsVoluntaryContextSwitches: 21 - TotalRawHdfsReadTime(*): 32395 - TotalReadThroughput: 220 - TotalTime: 18385435465 Averaged Fragment F01 split sizes: min: 256.00 MB, max: 303.37 MB, avg: 278.14 MB, stddev: 16.87 MB completion times: min:1m24s max:6m48s mean: 2m57s stddev:2m14s execution rates: min:697.11 KB/sec max:3.50 MB/sec mean:2.35 MB/sec stddev:1.12 MB/sec num instances: 4 - AverageThreadTokens: 1.705187137042598 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1666560 - PerHostPeakMemUsage: 11002190 - PrepareTime: 13733482 - RowsProduced: 25000 - TotalCpuTime: 133262262829 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 78632051006 - TotalStorageWaitTime: 127585726991 - TotalTime: 170895642886 DataStreamSender (dst_id=21) - BytesSent: 2393003 - InactiveTotalTime: 0 - NetworkThroughput(*): 21963 - OverallThroughput: 33225 - PeakMemoryUsage: 16384 - SerializeBatchTime: 473213551 - ThriftTransmitTime(*): 123960540620 - TotalTime: 79123584908 - UncompressedRowBatchSize: 5400428 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.0015625 - AverageScannerThreadConcurrency: 0.7373399715504979 - BytesRead: 1099411 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1048211 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1048211 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 3 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 1609216 - PerReadThreadRawHdfsThroughput: 188850182 - RemoteScanRanges: 0 - RowsRead: 25000 - RowsReturned: 25000 - RowsReturnedRate: 1628 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 59 - ScannerThreadsTotalWallClockTime: 133262271213 - MaterializeTupleTime(*): 4119681867 - ScannerThreadsSysTime: 1499500 - ScannerThreadsUserTime: 29495000 - ScannerThreadsVoluntaryContextSwitches: 106 - TotalRawHdfsReadTime(*): 152595729 - TotalReadThroughput: 10442 - TotalTime: 90138134480 Fragment F03 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8a (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2736096 - PerHostPeakMemUsage: 78443800 - PrepareTime: 32630128250 - RowsProduced: 100 - TotalCpuTime: 102102941861 - TotalNetworkReceiveTime: 693643877982 - TotalNetworkSendTime: 350184 - TotalStorageWaitTime: 0 - TotalTime: 727021073252 CodeGen - CodegenTime: 718377868 - CompileTime: 2776416067 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 65684947282 - PrepareTime: 31307419794 - TotalTime: 100117314953 DataStreamSender (dst_id=25) - BytesSent: 1803 - InactiveTotalTime: 0 - NetworkThroughput(*): 4644489 - OverallThroughput: 5967985 - PeakMemoryUsage: 4096 - SerializeBatchTime: 104209 - ThriftTransmitTime(*): 388202 - TotalTime: 302112 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 726283331840 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2342154 - GetNewBlockTime: 163948251 - GetResultsTime: 451886 - HTResizeTime: 961617 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2690016 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 974 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 726142286979 - UnpinTime: 20190 EXCHANGE_NODE (id=23) - BytesReceived: 48798 - ConvertRowBatchTime: 512355 - DeserializeRowBatchTimer: 825832 - FirstBatchArrivalWaitTime: 642479551124 - InactiveTotalTime: 693643806317 - PeakMemoryUsage: 0 - RowsReturned: 2654 - RowsReturnedRate: 3 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 693644451784 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e89 (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737848 - PerHostPeakMemUsage: 97477984 - PrepareTime: 32699962097 - RowsProduced: 100 - TotalCpuTime: 142494718167 - TotalNetworkReceiveTime: 653232104611 - TotalNetworkSendTime: 607322 - TotalStorageWaitTime: 0 - TotalTime: 686477667328 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4718608 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 463935749 - CompileTime: 1454390352 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 107524965676 - PrepareTime: 30252729414 - TotalTime: 139865891664 DataStreamSender (dst_id=25) - BytesSent: 1809 - InactiveTotalTime: 0 - NetworkThroughput(*): 3414470 - OverallThroughput: 9783455 - PeakMemoryUsage: 4096 - SerializeBatchTime: 92681 - ThriftTransmitTime(*): 529804 - TotalTime: 184904 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 685941547550 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2562010 - GetNewBlockTime: 687647173 - GetResultsTime: 440616 - HTResizeTime: 975455 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2691768 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1011 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 685712535798 - UnpinTime: 22925 EXCHANGE_NODE (id=23) - BytesReceived: 50910 - ConvertRowBatchTime: 513030 - DeserializeRowBatchTimer: 798183 - FirstBatchArrivalWaitTime: 602068852462 - InactiveTotalTime: 653231988796 - PeakMemoryUsage: 0 - RowsReturned: 2767 - RowsReturnedRate: 4 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 653232679831 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e88 (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2739048 - PerHostPeakMemUsage: 74082784 - PrepareTime: 68493131761 - RowsProduced: 100 - TotalCpuTime: 183390423542 - TotalNetworkReceiveTime: 612414901115 - TotalNetworkSendTime: 3901879 - TotalStorageWaitTime: 0 - TotalTime: 682115079015 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 96 - BlocksRecycled: 32 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4997120 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 851253847 - CompileTime: 1324901886 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 111652273807 - PrepareTime: 64085160836 - TotalTime: 178329830380 DataStreamSender (dst_id=25) - BytesSent: 1898 - InactiveTotalTime: 0 - NetworkThroughput(*): 480381 - OverallThroughput: 6589316 - PeakMemoryUsage: 4096 - SerializeBatchTime: 91711 - ThriftTransmitTime(*): 3951030 - TotalTime: 288042 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 680917832361 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2540465 - GetNewBlockTime: 1869434734 - GetResultsTime: 456544 - HTResizeTime: 1012807 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2692968 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1033 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 680260319585 - UnpinTime: 29555 EXCHANGE_NODE (id=23) - BytesReceived: 52142 - ConvertRowBatchTime: 619962 - DeserializeRowBatchTimer: 921350 - FirstBatchArrivalWaitTime: 561257991315 - InactiveTotalTime: 612414753993 - PeakMemoryUsage: 0 - RowsReturned: 2833 - RowsReturnedRate: 4 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 612415596178 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e87 (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737000 - PerHostPeakMemUsage: 85412376 - PrepareTime: 52137337941 - RowsProduced: 100 - TotalCpuTime: 173634133267 - TotalNetworkReceiveTime: 621945213609 - TotalNetworkSendTime: 386810 - TotalStorageWaitTime: 0 - TotalTime: 675022109388 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4997136 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 2206398171 - CompileTime: 2645071253 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 116726732702 - PrepareTime: 47324205380 - TotalTime: 167477681983 DataStreamSender (dst_id=25) - BytesSent: 1804 - InactiveTotalTime: 0 - NetworkThroughput(*): 4242649 - OverallThroughput: 7048499 - PeakMemoryUsage: 4096 - SerializeBatchTime: 84095 - ThriftTransmitTime(*): 425206 - TotalTime: 255941 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 674091810747 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2265213 - GetNewBlockTime: 1662292266 - GetResultsTime: 449966 - HTResizeTime: 807772 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2690920 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 990 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 673331230213 - UnpinTime: 22119 EXCHANGE_NODE (id=23) - BytesReceived: 50476 - ConvertRowBatchTime: 504175 - DeserializeRowBatchTimer: 810193 - FirstBatchArrivalWaitTime: 570788948551 - InactiveTotalTime: 621945133725 - PeakMemoryUsage: 0 - RowsReturned: 2729 - RowsReturnedRate: 4 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 621945776659 Fragment F00 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8e (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:16/1.44 GB - AverageThreadTokens: 1.4660514172709294 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 76059928 - PerHostPeakMemUsage: 78443800 - PrepareTime: 33189315604 - RowsProduced: 3685 - TotalCpuTime: 835822328142 - TotalNetworkReceiveTime: 278342131105 - TotalNetworkSendTime: 7148593 - TotalStorageWaitTime: 3001756787 - TotalTime: 674130850301 CodeGen - CodegenTime: 619075992 - CompileTime: 2786668671 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 85969789595 - PrepareTime: 29726652418 - TotalTime: 119122534203 DataStreamSender (dst_id=23) - BytesSent: 67946 - InactiveTotalTime: 0 - NetworkThroughput(*): 5005338 - OverallThroughput: 11515269 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2873608 - ThriftTransmitTime(*): 13574705 - TotalTime: 5900513 - UncompressedRowBatchSize: 199022 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 229357530 - GetResultsTime: 1082206 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25982208 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3685 - RowsReturnedRate: 5 - StreamingTime: 1491743682 - TotalTime: 672170450483 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24671488 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 639359645167 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 6283538982 - PeakMemoryUsage: 49160 - ProbeRows: 648271 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 8272120762 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6180682680716165 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 179176 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1988581780 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 9449 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 10889 - GetNewBlockTime: 161539115 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 2432235860 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24728848 - PinTime: 0 - ProbeRows: 4480754 - ProbeRowsPartitioned: 0 - ProbeTime: 823581486 - RowsReturned: 179176 - RowsReturnedRate: 284 - SpilledPartitions: 0 - TotalTime: 630154787127 - UnpinTime: 21868 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 4143 - DeserializeRowBatchTimer: 18195 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 29311 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 34116 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 2374118867 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 828145510 - GetNewBlockTime: 181071 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 12635439869 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 38286592 - PinTime: 0 - ProbeRows: 4480754 - ProbeRowsPartitioned: 0 - ProbeTime: 7442267494 - RowsReturned: 4480754 - RowsReturnedRate: 7138 - SpilledPartitions: 0 - TotalTime: 627722517151 - UnpinTime: 16889 EXCHANGE_NODE (id=21) - BytesReceived: 2393003 - ConvertRowBatchTime: 2268998718 - DeserializeRowBatchTimer: 3870363633 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 278314416056 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 356 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 280612129030 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24638720 - RowsReturned: 4274176 - RowsReturnedRate: 12778 - TotalTime: 334474948252 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15745096121 - PeakMemoryUsage: 24576 - ProbeRows: 4480754 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 234643459558 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 218898363437 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 4 - BuildTime: 9819624259 - InactiveTotalTime: 0 - LocalTime: 146482617676 - PeakMemoryUsage: 24576 - ProbeRows: 4650098 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 189365058934 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205953600603124 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1692522 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15195144414 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 51663457661 - PeakMemoryUsage: 16384 - ProbeRows: 4650098 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 67299914387 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.747437256354718 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1692522 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15636456726 UNNEST_NODE (id=4) - AvgCollectionSize: 6.71534451154191 - InactiveTotalTime: 0 - MaxCollectionSize: 25 - MinCollectionSize: 1 - NumCollections: 252038 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2892443640 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 16 Hdfs split stats (:<# splits>/): 0:16/1.44 GB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:128 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 263790162 - BytesReadDataNodeCache: 0 - BytesReadLocal: 263482962 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 263482962 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 16 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 51868928 - PerReadThreadRawHdfsThroughput: 364525090 - RemoteScanRanges: 3 - RowsRead: 49814903 - RowsReturned: 252038 - RowsReturnedRate: 2598 - ScanRangesComplete: 16 - ScannerThreadsInvoluntaryContextSwitches: 48299 - ScannerThreadsTotalWallClockTime: 353697238802 - MaterializeTupleTime(*): 347153237640 - ScannerThreadsSysTime: 378942000 - ScannerThreadsUserTime: 100951653000 - ScannerThreadsVoluntaryContextSwitches: 357 - TotalRawHdfsReadTime(*): 723654336 - TotalReadThroughput: 361435 - TotalTime: 96978183886 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8b (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:14/1.42 GB - AverageThreadTokens: 1.5062541145490453 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 83036696 - PerHostPeakMemUsage: 85412376 - PrepareTime: 51980348494 - RowsProduced: 3666 - TotalCpuTime: 963950395353 - TotalNetworkReceiveTime: 210846576883 - TotalNetworkSendTime: 5085750 - TotalStorageWaitTime: 5116374705 - TotalTime: 658261720746 CodeGen - CodegenTime: 1036455967 - CompileTime: 3137312751 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 132736990621 - PrepareTime: 42617451830 - TotalTime: 180761902777 DataStreamSender (dst_id=23) - BytesSent: 67446 - InactiveTotalTime: 0 - NetworkThroughput(*): 7239669 - OverallThroughput: 11154651 - PeakMemoryUsage: 163840 - SerializeBatchTime: 3450985 - ThriftTransmitTime(*): 9316171 - TotalTime: 6046446 - UncompressedRowBatchSize: 197996 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 940585593 - GetResultsTime: 1052423 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25982208 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3666 - RowsReturnedRate: 5 - StreamingTime: 1925470691 - TotalTime: 654790187888 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24671488 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 606090167322 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 8446802000 - PeakMemoryUsage: 49160 - ProbeRows: 640036 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 10023939220 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6185780918727914 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 176875 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1577137220 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 7496 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 5289 - GetNewBlockTime: 225754101 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 4934380936 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24728848 - PinTime: 0 - ProbeRows: 4413490 - ProbeRowsPartitioned: 0 - ProbeTime: 898486054 - RowsReturned: 176875 - RowsReturnedRate: 295 - SpilledPartitions: 0 - TotalTime: 597733746979 - UnpinTime: 16868 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3182 - DeserializeRowBatchTimer: 20839 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 41937 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 23845 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 3808059349 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 948759947 - GetNewBlockTime: 684172 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 18115301344 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 38286592 - PinTime: 0 - ProbeRows: 4413490 - ProbeRowsPartitioned: 0 - ProbeTime: 7393980719 - RowsReturned: 4413490 - RowsReturnedRate: 7445 - SpilledPartitions: 0 - TotalTime: 592799342198 - UnpinTime: 16361 EXCHANGE_NODE (id=21) - BytesReceived: 2393003 - ConvertRowBatchTime: 161828434 - DeserializeRowBatchTimer: 2459628885 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 210845681943 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 473 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 211009446158 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24638720 - RowsReturned: 4209664 - RowsReturnedRate: 11575 - TotalTime: 363674594696 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15303825976 - PeakMemoryUsage: 24576 - ProbeRows: 4413490 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 226592067820 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 211288241844 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 4 - BuildTime: 9804124110 - InactiveTotalTime: 0 - LocalTime: 139646685713 - PeakMemoryUsage: 24576 - ProbeRows: 4579880 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 182906955167 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205449388297857 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667233 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15089104497 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 51464570342 - PeakMemoryUsage: 16384 - ProbeRows: 4579880 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 66078360530 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.746994571244691 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667233 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14613790188 UNNEST_NODE (id=4) - AvgCollectionSize: 6.713239379907389 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248350 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2470344724 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 14 Hdfs split stats (:<# splits>/): 0:14/1.42 GB Hdfs Read Thread Concurrency Bucket: 0:99.61% 1:0.3901% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:112 - AverageHdfsReadThreadConcurrency: 0.0039011703511053317 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 255461223 - BytesReadDataNodeCache: 0 - BytesReadLocal: 255256423 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 255256423 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 14 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 58845696 - PerReadThreadRawHdfsThroughput: 362751765 - RemoteScanRanges: 2 - RowsRead: 49101937 - RowsReturned: 248350 - RowsReturnedRate: 1855 - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 44572 - ScannerThreadsTotalWallClockTime: 384417167484 - MaterializeTupleTime(*): 377692619860 - ScannerThreadsSysTime: 379942000 - ScannerThreadsUserTime: 102317445000 - ScannerThreadsVoluntaryContextSwitches: 424 - TotalRawHdfsReadTime(*): 704231508 - TotalReadThroughput: 355727 - TotalTime: 133868180347 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8c (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:17/1.41 GB - AverageThreadTokens: 1.4783491204330177 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 71707104 - PerHostPeakMemUsage: 74082784 - PrepareTime: 85120845504 - RowsProduced: 0 - TotalCpuTime: 924248246318 - TotalNetworkReceiveTime: 189815228801 - TotalNetworkSendTime: 4962 - TotalStorageWaitTime: 4751443774 - TotalTime: 637221447544 CodeGen - CodegenTime: 1255171363 - CompileTime: 5221420988 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 120796050698 - PrepareTime: 76496510167 - TotalTime: 203958662625 DataStreamSender (dst_id=23) - BytesSent: 0 - InactiveTotalTime: 0 - NetworkThroughput(*): 0 - OverallThroughput: 0 - PeakMemoryUsage: 16384 - SerializeBatchTime: 0 - ThriftTransmitTime(*): 0 - TotalTime: 98000 - UncompressedRowBatchSize: 0 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 676246287 - GetResultsTime: 0 - HTResizeTime: 0 - HashBuckets: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24933632 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - StreamingTime: 2871372203 - TotalTime: 634740600764 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 23622912 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 551952568095 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 59367241 - PeakMemoryUsage: 8200 - ProbeRows: 0 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 59428669 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 0.0 - InactiveTotalTime: 0 - MaxCollectionSize: 0 - MinCollectionSize: 0 - NumCollections: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 61428 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 2817 - BuildRows: 0 - BuildRowsPartitioned: 0 - BuildTime: 0 - GetNewBlockTime: 45669352 - HashBuckets: 0 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - LocalTime: 4613548028 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 23614720 - PinTime: 0 - ProbeRows: 4325832 - ProbeRowsPartitioned: 0 - ProbeTime: 1235298711 - RowsReturned: 0 - RowsReturnedRate: 0 - SpilledPartitions: 0 - TotalTime: 555141332474 - UnpinTime: 19406 EXCHANGE_NODE (id=22) - BytesReceived: 0 - ConvertRowBatchTime: 2060 - DeserializeRowBatchTimer: 0 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 17185 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 3296921620 - BuildRows: 97952 - BuildRowsPartitioned: 97952 - BuildTime: 706093527 - GetNewBlockTime: 216526 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 14788394916 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37238016 - PinTime: 0 - ProbeRows: 4416256 - ProbeRowsPartitioned: 0 - ProbeTime: 6439825346 - RowsReturned: 4325832 - RowsReturnedRate: 7857 - SpilledPartitions: 0 - TotalTime: 550527767261 - UnpinTime: 17016 EXCHANGE_NODE (id=21) - BytesReceived: 2347305 - ConvertRowBatchTime: 768396178 - DeserializeRowBatchTimer: 7451459355 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 189814347748 - PeakMemoryUsage: 0 - RowsReturned: 97952 - RowsReturnedRate: 513 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 190584601098 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 23590144 - RowsReturned: 4213760 - RowsReturnedRate: 12208 - TotalTime: 345154771247 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 13683611748 - PeakMemoryUsage: 24576 - ProbeRows: 4416256 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 216847732829 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 203164121081 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 6 - BuildTime: 10126929054 - InactiveTotalTime: 0 - LocalTime: 136596892709 - PeakMemoryUsage: 24576 - ProbeRows: 4583146 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 176596129441 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205374756109392 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1669806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15346121874 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 48232144570 - PeakMemoryUsage: 16384 - ProbeRows: 4583146 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 61973724954 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.7447176498347714 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1669806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 13741580384 UNNEST_NODE (id=4) - AvgCollectionSize: 6.711277059234906 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2294713910 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 0:17/1.41 GB Hdfs Read Thread Concurrency Bucket: 0:99.86% 1:0.1414% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:136 - AverageHdfsReadThreadConcurrency: 0.0014144271570014145 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 263557085 - BytesReadDataNodeCache: 0 - BytesReadLocal: 263454685 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 263454685 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 17 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 47622616 - PerReadThreadRawHdfsThroughput: 738900121 - RemoteScanRanges: 1 - RowsRead: 49173693 - RowsReturned: 248806 - RowsReturnedRate: 1977 - ScanRangesComplete: 17 - ScannerThreadsInvoluntaryContextSwitches: 47491 - ScannerThreadsTotalWallClockTime: 353978002041 - MaterializeTupleTime(*): 347521990889 - ScannerThreadsSysTime: 420936000 - ScannerThreadsUserTime: 103174315000 - ScannerThreadsVoluntaryContextSwitches: 457 - TotalRawHdfsReadTime(*): 356688377 - TotalReadThroughput: 385769 - TotalTime: 125815207818 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8d (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:14/1.41 GB - AverageThreadTokens: 1.452027027027027 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 95102304 - PerHostPeakMemUsage: 97477984 - PrepareTime: 45234715035 - RowsProduced: 3632 - TotalCpuTime: 840369649170 - TotalNetworkReceiveTime: 236139698198 - TotalNetworkSendTime: 5063837 - TotalStorageWaitTime: 2916751937 - TotalTime: 625755099884 CodeGen - CodegenTime: 800949720 - CompileTime: 5431830311 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 113334794365 - PrepareTime: 37876435175 - TotalTime: 157333844346 DataStreamSender (dst_id=23) - BytesSent: 66934 - InactiveTotalTime: 0 - NetworkThroughput(*): 4285052 - OverallThroughput: 10998964 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2786067 - ThriftTransmitTime(*): 15620344 - TotalTime: 6085482 - UncompressedRowBatchSize: 196160 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 222300 - GetResultsTime: 1091340 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24933632 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3632 - RowsReturnedRate: 5 - StreamingTime: 2567100239 - TotalTime: 623445008469 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 23622912 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 581501415882 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 7270557564 - PeakMemoryUsage: 49160 - ProbeRows: 629454 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 9191144961 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6116982820945362 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 174282 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1920587397 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 8075 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 5893 - GetNewBlockTime: 306373670 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 6226152032 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 23680272 - PinTime: 0 - ProbeRows: 4343863 - ProbeRowsPartitioned: 0 - ProbeTime: 1086859704 - RowsReturned: 174282 - RowsReturnedRate: 303 - SpilledPartitions: 0 - TotalTime: 573742917122 - UnpinTime: 26215 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3124 - DeserializeRowBatchTimer: 21782 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 37200 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 26881 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 2911382034 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 331925278 - GetNewBlockTime: 252813 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 15204257883 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37238016 - PinTime: 0 - ProbeRows: 4343863 - ProbeRowsPartitioned: 0 - ProbeTime: 7620107247 - RowsReturned: 4343863 - RowsReturnedRate: 7654 - SpilledPartitions: 0 - TotalTime: 567516738209 - UnpinTime: 16764 EXCHANGE_NODE (id=21) - BytesReceived: 2393003 - ConvertRowBatchTime: 1910038242 - DeserializeRowBatchTimer: 5576389283 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 236138690764 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 419 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 238580435817 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 23590144 - RowsReturned: 4145152 - RowsReturnedRate: 13212 - TotalTime: 313732044509 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15794495229 - PeakMemoryUsage: 24576 - ProbeRows: 4343863 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 234662401961 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 218867906732 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 2 - BuildTime: 10091928946 - InactiveTotalTime: 0 - LocalTime: 146249270179 - PeakMemoryUsage: 24576 - ProbeRows: 4508276 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 188833742350 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205691209456636 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1641514 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15323669664 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 51980404859 - PeakMemoryUsage: 16384 - ProbeRows: 4508276 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 66678340433 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.746413372045563 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1641514 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14697935574 UNNEST_NODE (id=4) - AvgCollectionSize: 6.698744735725246 - InactiveTotalTime: 0 - MaxCollectionSize: 25 - MinCollectionSize: 1 - NumCollections: 245048 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 3043252485 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 14 Hdfs split stats (:<# splits>/): 0:14/1.41 GB Hdfs Read Thread Concurrency Bucket: 0:99.85% 1:0.1493% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:112 - AverageHdfsReadThreadConcurrency: 0.0014925373134328358 - AverageScannerThreadConcurrency: 0.9985074626865672 - BytesRead: 252141413 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251834213 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251834213 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 14 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 70911304 - PerReadThreadRawHdfsThroughput: 428476932 - RemoteScanRanges: 3 - RowsRead: 48367623 - RowsReturned: 245048 - RowsReturnedRate: 3226 - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 48060 - ScannerThreadsTotalWallClockTime: 334816742813 - MaterializeTupleTime(*): 330545194426 - ScannerThreadsSysTime: 437933000 - ScannerThreadsUserTime: 102572406000 - ScannerThreadsVoluntaryContextSwitches: 363 - TotalRawHdfsReadTime(*): 588459714 - TotalReadThroughput: 358856 - TotalTime: 75957417429 Fragment F02 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e8f (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:1/4.17 KB - AverageThreadTokens: 1.2278481012658229 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 73024 - PerHostPeakMemUsage: 7695240 - PrepareTime: 259507 - RowsProduced: 1 - TotalCpuTime: 25176618896 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 59531281644 - TotalStorageWaitTime: 15099898078 - TotalTime: 20065395070 DataStreamSender (dst_id=22) - BytesSent: 164 - InactiveTotalTime: 0 - NetworkThroughput(*): 1 - OverallThroughput: 4512 - PeakMemoryUsage: 21824 - SerializeBatchTime: 36233182 - ThriftTransmitTime(*): 121445498990 - TotalTime: 36343092 - UncompressedRowBatchSize: 176 HDFS_SCAN_NODE (id=12) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 1 Hdfs split stats (:<# splits>/): 0:1/4.17 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:2 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 4758 - BytesReadDataNodeCache: 0 - BytesReadLocal: 4758 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 4758 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 43008 - PerReadThreadRawHdfsThroughput: 146874517 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 18132821884 - MaterializeTupleTime(*): 70045 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 999000 - ScannerThreadsVoluntaryContextSwitches: 21 - TotalRawHdfsReadTime(*): 32395 - TotalReadThroughput: 220 - TotalTime: 18385435465 Fragment F01 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e90 (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:3/278.37 MB - AverageThreadTokens: 1.769436997319035 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1034240 - PerHostPeakMemUsage: 18715528 - PrepareTime: 268449 - RowsProduced: 25022 - TotalCpuTime: 321053345405 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 121527275644 - TotalStorageWaitTime: 317557621026 - TotalTime: 406997420861 DataStreamSender (dst_id=21) - BytesSent: 2353752 - InactiveTotalTime: 0 - NetworkThroughput(*): 18693 - OverallThroughput: 19183 - PeakMemoryUsage: 16384 - SerializeBatchTime: 1160407348 - ThriftTransmitTime(*): 125912541460 - TotalTime: 122695958823 - UncompressedRowBatchSize: 5405168 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 3 Hdfs split stats (:<# splits>/): 0:3/278.37 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:9 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.7756756756756756 - BytesRead: 1074417 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1074417 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1074417 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 3 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 976896 - PerReadThreadRawHdfsThroughput: 251011717 - RemoteScanRanges: 0 - RowsRead: 25022 - RowsReturned: 25022 - RowsReturnedRate: 88 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 46 - ScannerThreadsTotalWallClockTime: 321053354487 - MaterializeTupleTime(*): 1626500660 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 28995000 - ScannerThreadsVoluntaryContextSwitches: 69 - TotalRawHdfsReadTime(*): 4280346 - TotalReadThroughput: 2871 - TotalTime: 282170874777 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e92 (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AverageThreadTokens: 1.7830188679245282 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1298432 - PerHostPeakMemUsage: 9120928 - PrepareTime: 235164 - RowsProduced: 23012 - TotalCpuTime: 104963003087 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 49672912334 - TotalStorageWaitTime: 90842550364 - TotalTime: 109094231543 DataStreamSender (dst_id=21) - BytesSent: 2165716 - InactiveTotalTime: 0 - NetworkThroughput(*): 36309 - OverallThroughput: 43436 - PeakMemoryUsage: 16384 - SerializeBatchTime: 179998497 - ThriftTransmitTime(*): 59646505975 - TotalTime: 49859116960 - UncompressedRowBatchSize: 4970976 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 2 Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:6 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.8736842105263158 - BytesRead: 910178 - BytesReadDataNodeCache: 0 - BytesReadLocal: 910178 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 910178 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 2 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 1241088 - PerReadThreadRawHdfsThroughput: 260058138 - RemoteScanRanges: 0 - RowsRead: 23012 - RowsReturned: 23012 - RowsReturnedRate: 397 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 43 - ScannerThreadsTotalWallClockTime: 104963012769 - MaterializeTupleTime(*): 12121043578 - ScannerThreadsSysTime: 4999000 - ScannerThreadsUserTime: 23996000 - ScannerThreadsVoluntaryContextSwitches: 159 - TotalRawHdfsReadTime(*): 3499902 - TotalReadThroughput: 9402 - TotalTime: 57889996318 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e93 (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:4/274.81 MB - AverageThreadTokens: 1.920731707317073 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2060288 - PerHostPeakMemUsage: 7989664 - PrepareTime: 328664 - RowsProduced: 24695 - TotalCpuTime: 77581021259 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 67354630832 - TotalStorageWaitTime: 75379007419 - TotalTime: 83844503164 DataStreamSender (dst_id=21) - BytesSent: 2441812 - InactiveTotalTime: 0 - NetworkThroughput(*): 17703 - OverallThroughput: 35868 - PeakMemoryUsage: 16384 - SerializeBatchTime: 528365331 - ThriftTransmitTime(*): 137924480307 - TotalTime: 68076765197 - UncompressedRowBatchSize: 5334552 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 4 Hdfs split stats (:<# splits>/): 0:4/274.81 MB Hdfs Read Thread Concurrency Bucket: 0:99.38% 1:0.625% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.00625 - AverageScannerThreadConcurrency: 0.94375 - BytesRead: 1167078 - BytesReadDataNodeCache: 0 - BytesReadLocal: 962278 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 962278 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 4 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 2002944 - PerReadThreadRawHdfsThroughput: 1953392 - RemoteScanRanges: 2 - RowsRead: 24695 - RowsReturned: 24695 - RowsReturnedRate: 1748 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 64 - ScannerThreadsTotalWallClockTime: 77581026677 - MaterializeTupleTime(*): 1171433956 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 32994000 - ScannerThreadsVoluntaryContextSwitches: 92 - TotalRawHdfsReadTime(*): 597462044 - TotalReadThroughput: 14222 - TotalTime: 14123388313 Instance 4d46f4c42facb2e7:50aa3b04b8bd0e91 (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:4/303.37 MB - AverageThreadTokens: 1.3475609756097562 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2273280 - PerHostPeakMemUsage: 8182640 - PrepareTime: 54101653 - RowsProduced: 27271 - TotalCpuTime: 29451681567 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 75973385217 - TotalStorageWaitTime: 26563729157 - TotalTime: 83646415976 DataStreamSender (dst_id=21) - BytesSent: 2610732 - InactiveTotalTime: 0 - NetworkThroughput(*): 15147 - OverallThroughput: 34413 - PeakMemoryUsage: 16384 - SerializeBatchTime: 24083029 - ThriftTransmitTime(*): 172358634740 - TotalTime: 75862498654 - UncompressedRowBatchSize: 5891016 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 4 Hdfs split stats (:<# splits>/): 0:4/303.37 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.35625 - BytesRead: 1245972 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1245972 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1245972 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 4 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 2215936 - PerReadThreadRawHdfsThroughput: 242377484 - RemoteScanRanges: 0 - RowsRead: 27271 - RowsReturned: 27271 - RowsReturnedRate: 4282 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 86 - ScannerThreadsTotalWallClockTime: 29451690921 - MaterializeTupleTime(*): 1559749276 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 31995000 - ScannerThreadsVoluntaryContextSwitches: 106 - TotalRawHdfsReadTime(*): 5140626 - TotalReadThroughput: 15274 - TotalTime: 6368278514