Query (id=6643d7f56f3d4163:bb041e84a766fd95) Summary Session ID: 5a4e5efa88ad6309:ebedfaf0b49bea90 Session Type: HIVESERVER2 HiveServer2 Protocol Version: V6 Start Time: 2016-02-23 14:16:57.836993000 End Time: 2016-02-23 14:37:32.767048000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.5.0-cdh5.7.0 DEBUG (build 22f71f3e82151c1064bf2983dd3e9025c4cce5b3) User: jenkins Connected User: jenkins Delegated User: Network Address: 172.26.26.3:55926 Default Db: tpch_10_nested_parquet Sql Statement: /* Mem: 514 MB. Coordinator: impala-stress-cdh570-2.vpc.cloudera.com. */ select s_name, count(*) as numwait from supplier s, customer c, c.c_orders o, o.o_lineitems l1, region.r_nations n where s_suppkey = l1.l_suppkey and o_orderstatus = 'F' and l1.l_receiptdate > l1.l_commitdate and exists ( select * from o.o_lineitems l2 where l2.l_suppkey <> l1.l_suppkey ) and not exists ( select * from o.o_lineitems l3 where l3.l_suppkey <> l1.l_suppkey and l3.l_receiptdate > l3.l_commitdate ) and s_nationkey = n_nationkey and n_name = 'SAUDI ARABIA' group by s_name order by numwait desc, s_name limit 100 Coordinator: impala-stress-cdh570-2.vpc.cloudera.com:22000 Query Options (non default): ABORT_ON_ERROR=1,MEM_LIMIT=538968064,RUNTIME_FILTER_MODE=0,DISABLE_ROW_RUNTIME_FILTERING=1 Plan: ---------------- Estimated Per-Host Requirements: Memory=300.62MB VCores=2 WARNING: The following tables are missing relevant table and/or column statistics. tpch_10_nested_parquet.customer, tpch_10_nested_parquet.region 25:MERGING-EXCHANGE [UNPARTITIONED] | order by: count(*) DESC, s_name ASC | limit: 100 | hosts=4 per-host-mem=unavailable | tuple-ids=10 row-size=42B cardinality=100 | 20:TOP-N [LIMIT=100] | order by: count(*) DESC, s_name ASC | hosts=4 per-host-mem=4.10KB | tuple-ids=10 row-size=42B cardinality=100 | 24:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: s_name | hosts=4 per-host-mem=10.00MB | tuple-ids=9 row-size=42B cardinality=96988 | 23:EXCHANGE [HASH(s_name)] | hosts=4 per-host-mem=0B | tuple-ids=9 row-size=42B cardinality=96988 | 19:AGGREGATE [STREAMING] | output: count(*) | group by: s_name | hosts=4 per-host-mem=10.00MB | tuple-ids=9 row-size=42B cardinality=96988 | 18:SUBPLAN | hosts=4 per-host-mem=0B | tuple-ids=3,2,1,0,4 row-size=182B cardinality=150000000 | |--16:NESTED LOOP JOIN [RIGHT ANTI JOIN] | | join predicates: l3.l_suppkey != l1.l_suppkey | | hosts=4 per-host-mem=182B | | tuple-ids=3,2,1,0,4 row-size=182B cardinality=1 | | | |--14:SINGULAR ROW SRC | | parent-subplan=18 | | hosts=4 per-host-mem=0B | | tuple-ids=3,2,1,0,4 row-size=182B cardinality=1 | | | 15:UNNEST [o.o_lineitems l3] | parent-subplan=18 | hosts=4 per-host-mem=0B | tuple-ids=7 row-size=40B cardinality=10 | 17:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: s_nationkey = n_nationkey | hosts=4 per-host-mem=100B | tuple-ids=3,2,1,0,4 row-size=182B cardinality=150000000 | |--22:EXCHANGE [BROADCAST] | | hosts=1 per-host-mem=0B | | tuple-ids=4 row-size=18B cardinality=5 | | | 12:SCAN HDFS [tpch_10_nested_parquet.region.r_nations n, RANDOM] | partitions=1/1 files=1 size=4.17KB | predicates: n_name = 'SAUDI ARABIA' | table stats: 5 rows total | column stats: all | hosts=1 per-host-mem=32.00MB | tuple-ids=4 row-size=18B cardinality=5 | 13:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: l1.l_suppkey = s_suppkey | hosts=4 per-host-mem=4.62MB | tuple-ids=3,2,1,0 row-size=164B cardinality=150000000 | |--21:EXCHANGE [BROADCAST] | | hosts=4 per-host-mem=0B | | tuple-ids=0 row-size=44B cardinality=100000 | | | 00:SCAN HDFS [tpch_10_nested_parquet.supplier s, RANDOM] | partitions=1/1 files=5 size=1.09GB | table stats: 100000 rows total | column stats: all | hosts=4 per-host-mem=264.00MB | tuple-ids=0 row-size=44B cardinality=100000 | 02:SUBPLAN | hosts=4 per-host-mem=0B | tuple-ids=3,2,1 row-size=120B cardinality=150000000 | |--11:NESTED LOOP JOIN [CROSS JOIN] | | hosts=4 per-host-mem=16B | | tuple-ids=3,2,1 row-size=120B cardinality=100 | | | |--03:SINGULAR ROW SRC | | parent-subplan=02 | | hosts=4 per-host-mem=0B | | tuple-ids=1 row-size=16B cardinality=1 | | | 05:SUBPLAN | | hosts=4 per-host-mem=0B | | tuple-ids=3,2 row-size=104B cardinality=100 | | | |--10:NESTED LOOP JOIN [LEFT SEMI JOIN] | | | join predicates: l2.l_suppkey != l1.l_suppkey | | | hosts=4 per-host-mem=80B | | | tuple-ids=3,2 row-size=104B cardinality=10 | | | | | |--08:UNNEST [o.o_lineitems l2] | | | parent-subplan=05 | | | hosts=4 per-host-mem=0B | | | tuple-ids=5 row-size=8B cardinality=10 | | | | | 09:NESTED LOOP JOIN [CROSS JOIN] | | | hosts=4 per-host-mem=64B | | | tuple-ids=3,2 row-size=104B cardinality=10 | | | | | |--06:SINGULAR ROW SRC | | | parent-subplan=05 | | | hosts=4 per-host-mem=0B | | | tuple-ids=2 row-size=64B cardinality=1 | | | | | 07:UNNEST [o.o_lineitems l1] | | parent-subplan=05 | | hosts=4 per-host-mem=0B | | tuple-ids=3 row-size=0B cardinality=10 | | | 04:UNNEST [c.c_orders o] | parent-subplan=02 | hosts=4 per-host-mem=0B | tuple-ids=2 row-size=0B cardinality=10 | 01:SCAN HDFS [tpch_10_nested_parquet.customer c, RANDOM] partitions=1/1 files=31 size=5.68GB predicates: !empty(c.c_orders) predicates on o: !empty(o.o_lineitems), o_orderstatus = 'F' predicates on l1: l1.l_receiptdate > l1.l_commitdate predicates on l3: l3.l_receiptdate > l3.l_commitdate table stats: 1500000 rows total column stats: unavailable hosts=4 per-host-mem=88.00MB tuple-ids=1 row-size=16B cardinality=1500000 ---------------- Estimated Per-Host Mem: 315218596 Estimated Per-Host VCores: 2 Tables Missing Stats: tpch_10_nested_parquet.customer,tpch_10_nested_parquet.region Request Pool: default-pool ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ------------------------------------------------------------------------------------------------------------------------------------------ 25:MERGING-EXCHANGE 1 128.305ms 128.305ms 400 100 0 -1.00 B UNPARTITIONED 20:TOP-N 4 305.416ms 706.126ms 400 100 20.00 KB 4.10 KB 24:AGGREGATE 4 43s906ms 1m 4.01K 96.99K 2.57 MB 10.00 MB FINALIZE 23:EXCHANGE 4 942.438us 1.012ms 14.63K 96.99K 0 0 HASH(s_name) 19:AGGREGATE 4 52s173ms 1m21s 14.63K 96.99K 24.78 MB 10.00 MB STREAMING 18:SUBPLAN 4 262.121ms 792.540ms 0 150.00M 23.53 MB 0 |--16:NESTED LOOP JOIN 4 6s400ms 6s744ms 0 1 48.01 KB 182.00 B RIGHT ANTI JOIN | |--14:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | 15:UNNEST 4 1s583ms 1s703ms 0 10 0 0 o.o_lineitems l3 17:HASH JOIN 4 4s225ms 5s899ms 705.46K 150.00M 23.58 MB 100.00 B INNER JOIN, BROADCAST |--22:EXCHANGE 4 27.269us 29.502us 1 5 0 0 BROADCAST | 12:SCAN HDFS 1 18s348ms 18s348ms 1 5 42.00 KB 32.00 MB tpch_10_nested_parquet.regi... 13:HASH JOIN 4 15s332ms 17s722ms 17.61M 150.00M 36.51 MB 4.62 MB INNER JOIN, BROADCAST |--21:EXCHANGE 4 784.879ms 1s416ms 100.00K 100.00K 0 0 BROADCAST | 00:SCAN HDFS 4 1m31s 4m38s 100.00K 100.00K 2.18 MB 264.00 MB tpch_10_nested_parquet.supp... 02:SUBPLAN 4 2s776ms 3s011ms 16.84M 150.00M 23.50 MB 0 |--11:NESTED LOOP JOIN 4 15s040ms 16s181ms 0 100 24.00 KB 16.00 B CROSS JOIN | |--03:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | 05:SUBPLAN 4 25s572ms 27s409ms 0 100 8.00 KB 0 | |--10:NESTED LOOP JOIN 4 2m20s 2m27s 0 10 24.00 KB 80.00 B LEFT SEMI JOIN | | |--08:UNNEST 4 14s792ms 16s054ms 0 10 0 0 o.o_lineitems l2 | | 09:NESTED LOOP JOIN 4 48s871ms 51s888ms 0 10 16.00 KB 64.00 B CROSS JOIN | | |--06:SINGULAR ROW SRC 4 0.000ns 0.000ns 0 1 0 0 | | 07:UNNEST 4 14s399ms 15s531ms 0 10 0 0 o.o_lineitems l1 | 04:UNNEST 4 2s657ms 2s933ms 0 10 0 0 c.c_orders o 01:SCAN HDFS 4 1m54s 2m13s 994.24K 1.50M 59.95 MB 88.00 MB tpch_10_nested_parquet.cust... Planner Timeline Analysis finished: 412346118 Equivalence classes computed: 629537439 Single node plan created: 1089511521 Distributed plan created: 5058937876 Lineage info computed: 5059864036 Planning finished: 5170667447 Query Timeline Start execution: 15926384 Planning finished: 5723730483 Ready to start 13 remote fragments: 5978553682 All 13 remote fragments started: 442376438560 Rows available: 1234763427974 First row fetched: 1234855295597 Unregister query: 1234885938933 ImpalaServer - ClientFetchWaitTimer: 116593539 - InactiveTotalTime: 0 - RowMaterializationTimer: 117506 - TotalTime: 0 Execution Profile 6643d7f56f3d4163:bb041e84a766fd95 Fragment start latencies: Count: 13, 25th %-ile: 265ms, 50th %-ile: 363ms, 75th %-ile: 417ms, 90th %-ile: 682ms, 95th %-ile: 682ms, 99.9th %-ile: 826ms Per Node Peak Memory Usage: impala-stress-cdh570-5.vpc.cloudera.com:22000(85.29 MB) impala-stress-cdh570-4.vpc.cloudera.com:22000(73.46 MB) impala-stress-cdh570-3.vpc.cloudera.com:22000(82.46 MB) impala-stress-cdh570-2.vpc.cloudera.com:22000(73.46 MB) - FiltersReceived: 0 - FinalizationTimer: 0 - InactiveTotalTime: 0 - TotalTime: 1228525839687 Coordinator Fragment F04 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 31392 - PerHostPeakMemUsage: 0 - PrepareTime: 89153 - RowsProduced: 100 - TotalCpuTime: 1228985486956 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 0 - TotalStorageWaitTime: 0 - TotalTime: 791875346673 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4997136 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 EXCHANGE_NODE (id=25) - BytesReceived: 7429 - ConvertRowBatchTime: 0 - DeserializeRowBatchTimer: 171992 - FirstBatchArrivalWaitTime: 791728238544 - InactiveTotalTime: 791728251615 - MergeGetNext: 807712 - MergeGetNextBatch: 50986 - PeakMemoryUsage: 0 - RowsReturned: 400 - RowsReturnedRate: 0 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 791856557322 Averaged Fragment F03 split sizes: min: 0, max: 0, avg: 0, stddev: 0 completion times: min:13m12s max:13m13s mean: 13m12s stddev:223.207ms execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec num instances: 4 - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737498 - PerHostPeakMemUsage: 82486482 - PrepareTime: 44202310881 - RowsProduced: 100 - TotalCpuTime: 142385365179 - TotalNetworkReceiveTime: 650569233547 - TotalNetworkSendTime: 602965 - TotalStorageWaitTime: 0 - TotalTime: 695437306530 CodeGen - CodegenTime: 944455681 - CompileTime: 2119223123 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 94506186484 - PrepareTime: 41565835138 - TotalTime: 139093475589 DataStreamSender (dst_id=25) - BytesSent: 1857 - InactiveTotalTime: 0 - NetworkThroughput(*): 4432661 - OverallThroughput: 7208059 - PeakMemoryUsage: 4096 - SerializeBatchTime: 92678 - ThriftTransmitTime(*): 655325 - TotalTime: 286932 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 694781622988 AGGREGATION_NODE (id=24) - BuildTime: 2652511 - GetNewBlockTime: 573014598 - GetResultsTime: 464376 - HTResizeTime: 872242 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2691418 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1002 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 694476206033 - UnpinTime: 20947 EXCHANGE_NODE (id=23) - BytesReceived: 67381 - ConvertRowBatchTime: 724241 - DeserializeRowBatchTimer: 1147640 - FirstBatchArrivalWaitTime: 605790802400 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 3657 - RowsReturnedRate: 5 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 650570046830 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4811450 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 Averaged Fragment F00 split sizes: min: 1.41 GB, max: 1.44 GB, avg: 1.42 GB, stddev: 14.11 MB completion times: min:12m28s max:13m12s mean: 12m50s stddev:16s029ms execution rates: min:1.83 MB/sec max:1.94 MB/sec mean:1.89 MB/sec stddev:40.15 KB/sec num instances: 4 - AverageThreadTokens: 1.47801385608148 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 80108754 - PerHostPeakMemUsage: 82486482 - PrepareTime: 53273950821 - RowsProduced: 3657 - TotalCpuTime: 893332446536 - TotalNetworkReceiveTime: 233459796991 - TotalNetworkSendTime: 11197428 - TotalStorageWaitTime: 3755300909 - TotalTime: 657071896287 CodeGen - CodegenTime: 715099498 - CompileTime: 4369398305 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 108273081569 - PrepareTime: 47441567038 - TotalTime: 161018397197 DataStreamSender (dst_id=23) - BytesSent: 67381 - InactiveTotalTime: 0 - NetworkThroughput(*): 4055716 - OverallThroughput: 11443188 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2683638 - ThriftTransmitTime(*): 19854408 - TotalTime: 5934255 - UncompressedRowBatchSize: 197510 AGGREGATION_NODE (id=19) - GetNewBlockTime: 220312530 - GetResultsTime: 1072925 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25457920 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3657 - RowsReturnedRate: 5 - StreamingTime: 2743783352 - TotalTime: 654751743209 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24147200 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 602577831865 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 6400441056 - PeakMemoryUsage: 49160 - ProbeRows: 637603 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 7983722102 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.615209001387032 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 176365 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1583281046 HASH_JOIN_NODE (id=17) - BuildPartitionTime: 7908 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 8061 - GetNewBlockTime: 439261325 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 4225604920 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24204560 - PinTime: 0 - ProbeRows: 4402269 - ProbeRowsPartitioned: 0 - ProbeTime: 1037984829 - RowsReturned: 176365 - RowsReturnedRate: 297 - SpilledPartitions: 0 - TotalTime: 594734715607 - UnpinTime: 21476 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3415 - DeserializeRowBatchTimer: 20942 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 36769 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 27269 HASH_JOIN_NODE (id=13) - BuildPartitionTime: 4993945209 - BuildRows: 99744 - BuildRowsPartitioned: 99744 - BuildTime: 573788346 - GetNewBlockTime: 73166792 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 15332470086 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37762304 - PinTime: 0 - ProbeRows: 4413590 - ProbeRowsPartitioned: 0 - ProbeTime: 6467597937 - RowsReturned: 4402269 - RowsReturnedRate: 7480 - SpilledPartitions: 0 - TotalTime: 590509083417 - UnpinTime: 17255 EXCHANGE_NODE (id=21) - BytesReceived: 2352501 - ConvertRowBatchTime: 611814379 - DeserializeRowBatchTimer: 6270001619 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 99744 - RowsReturnedRate: 434 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 234243695621 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24114432 - RowsReturned: 4210688 - RowsReturnedRate: 12373 - TotalTime: 340932917710 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15040736334 - PeakMemoryUsage: 24576 - ProbeRows: 4413590 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 224143841396 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 209103105061 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 2 - BuildTime: 9878969085 - InactiveTotalTime: 0 - LocalTime: 140582600739 - PeakMemoryUsage: 24576 - ProbeRows: 4580350 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 180873139741 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205617238616757 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667768 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14792319899 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 48871318558 - PeakMemoryUsage: 16384 - ProbeRows: 4580350 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 63270759501 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.746390712369937 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667768 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14399440942 UNNEST_NODE (id=4) - AvgCollectionSize: 6.709651421602362 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248560 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2657337239 HDFS_SCAN_NODE (id=1) - AverageHdfsReadThreadConcurrency: 0.0010403722281330703 - AverageScannerThreadConcurrency: 0.9996719160104987 - BytesRead: 258737470 - BytesReadDataNodeCache: 0 - BytesReadLocal: 258507070 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 258507070 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 15 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 55917754 - PerReadThreadRawHdfsThroughput: 465301954 - RemoteScanRanges: 2 - RowsRead: 49114539 - RowsReturned: 248560 - RowsReturnedRate: 2277 - ScanRangesComplete: 15 - ScannerThreadsInvoluntaryContextSwitches: 46666 - ScannerThreadsTotalWallClockTime: 360107080087 - MaterializeTupleTime(*): 354403629983 - ScannerThreadsSysTime: 558664750 - ScannerThreadsUserTime: 102042736500 - ScannerThreadsVoluntaryContextSwitches: 395 - TotalRawHdfsReadTime(*): 564377615 - TotalReadThroughput: 363985 - TotalTime: 114012425753 Averaged Fragment F02 split sizes: min: 4.17 KB, max: 4.17 KB, avg: 4.17 KB, stddev: 0 completion times: min:1m22s max:1m22s mean: 1m22s stddev:0.000ns execution rates: min:52.00 B/sec max:52.00 B/sec mean:52.00 B/sec stddev:0.00 /sec num instances: 1 - AverageThreadTokens: 1.2384105960264902 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 73024 - PerHostPeakMemUsage: 7302872 - PrepareTime: 375623 - RowsProduced: 1 - TotalCpuTime: 23275256541 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 58031767132 - TotalStorageWaitTime: 14813344730 - TotalTime: 19896548298 DataStreamSender (dst_id=22) - BytesSent: 164 - InactiveTotalTime: 0 - NetworkThroughput(*): 1 - OverallThroughput: 592 - PeakMemoryUsage: 21824 - SerializeBatchTime: 276540763 - ThriftTransmitTime(*): 123320861267 - TotalTime: 276667567 - UncompressedRowBatchSize: 176 HDFS_SCAN_NODE (id=12) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 4758 - BytesReadDataNodeCache: 0 - BytesReadLocal: 4758 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 4758 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 43008 - PerReadThreadRawHdfsThroughput: 130750206 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 18082768188 - MaterializeTupleTime(*): 77953 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 21 - TotalRawHdfsReadTime(*): 36390 - TotalReadThroughput: 226 - TotalTime: 18348243113 Averaged Fragment F01 split sizes: min: 256.00 MB, max: 303.37 MB, avg: 278.14 MB, stddev: 16.87 MB completion times: min:1m24s max:6m49s mean: 2m57s stddev:2m15s execution rates: min:696.16 KB/sec max:3.56 MB/sec mean:2.36 MB/sec stddev:1.14 MB/sec num instances: 4 - AverageThreadTokens: 1.7161849084840664 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1649152 - PerHostPeakMemUsage: 10930820 - PrepareTime: 41799436 - RowsProduced: 25000 - TotalCpuTime: 133739933024 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 76988927649 - TotalStorageWaitTime: 128157998233 - TotalTime: 170225244618 DataStreamSender (dst_id=21) - BytesSent: 2358602 - InactiveTotalTime: 0 - NetworkThroughput(*): 21548 - OverallThroughput: 34962 - PeakMemoryUsage: 16384 - SerializeBatchTime: 200088225 - ThriftTransmitTime(*): 125546132882 - TotalTime: 77198895587 - UncompressedRowBatchSize: 5400428 HDFS_SCAN_NODE (id=0) - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.7514160762337934 - BytesRead: 1099411 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1048211 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1048211 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 3 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 1591808 - PerReadThreadRawHdfsThroughput: 119066910 - RemoteScanRanges: 0 - RowsRead: 25000 - RowsReturned: 25000 - RowsReturnedRate: 1720 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 33 - ScannerThreadsTotalWallClockTime: 133739941034 - MaterializeTupleTime(*): 4709033304 - ScannerThreadsSysTime: 499500 - ScannerThreadsUserTime: 30744750 - ScannerThreadsVoluntaryContextSwitches: 87 - TotalRawHdfsReadTime(*): 143071726 - TotalReadThroughput: 10659 - TotalTime: 91320681200 Fragment F03 Instance 6643d7f56f3d4163:bb041e84a766fd9a (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2736096 - PerHostPeakMemUsage: 77031944 - PrepareTime: 31892022113 - RowsProduced: 100 - TotalCpuTime: 100923141638 - TotalNetworkReceiveTime: 692178217366 - TotalNetworkSendTime: 2133 - TotalStorageWaitTime: 0 - TotalTime: 724673330760 CodeGen - CodegenTime: 430712928 - CompileTime: 1036986988 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 66935413722 - PrepareTime: 30031205431 - TotalTime: 98757499040 DataStreamSender (dst_id=25) - BytesSent: 1845 - InactiveTotalTime: 0 - NetworkThroughput(*): 8202807 - OverallThroughput: 4130640 - PeakMemoryUsage: 4096 - SerializeBatchTime: 91466 - ThriftTransmitTime(*): 224923 - TotalTime: 446662 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 724079634624 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2182480 - GetNewBlockTime: 290614807 - GetResultsTime: 439439 - HTResizeTime: 534207 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2690016 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 974 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 723575835235 - UnpinTime: 23040 EXCHANGE_NODE (id=23) - BytesReceived: 65013 - ConvertRowBatchTime: 650878 - DeserializeRowBatchTimer: 1028281 - FirstBatchArrivalWaitTime: 647393679071 - InactiveTotalTime: 692178115812 - PeakMemoryUsage: 0 - RowsReturned: 3533 - RowsReturnedRate: 5 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 692178949199 Instance 6643d7f56f3d4163:bb041e84a766fd98 (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2739048 - PerHostPeakMemUsage: 77023760 - PrepareTime: 61383208761 - RowsProduced: 100 - TotalCpuTime: 147527434714 - TotalNetworkReceiveTime: 645431490586 - TotalNetworkSendTime: 489705 - TotalStorageWaitTime: 0 - TotalTime: 707396462822 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4718608 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 2067167877 - CompileTime: 1887250904 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 82827512103 - PrepareTime: 57610707013 - TotalTime: 143300762742 DataStreamSender (dst_id=25) - BytesSent: 1848 - InactiveTotalTime: 0 - NetworkThroughput(*): 3444824 - OverallThroughput: 6913787 - PeakMemoryUsage: 4096 - SerializeBatchTime: 92165 - ThriftTransmitTime(*): 536457 - TotalTime: 267292 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 706825217790 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2960452 - GetNewBlockTime: 860775583 - GetResultsTime: 520718 - HTResizeTime: 1016618 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2692968 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1034 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 706119091444 - UnpinTime: 20530 EXCHANGE_NODE (id=23) - BytesReceived: 69657 - ConvertRowBatchTime: 715117 - DeserializeRowBatchTimer: 1249249 - FirstBatchArrivalWaitTime: 600653929069 - InactiveTotalTime: 645431361300 - PeakMemoryUsage: 0 - RowsReturned: 3787 - RowsReturnedRate: 5 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 645432290799 Instance 6643d7f56f3d4163:bb041e84a766fd99 (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737848 - PerHostPeakMemUsage: 89429272 - PrepareTime: 33944958930 - RowsProduced: 100 - TotalCpuTime: 142860193370 - TotalNetworkReceiveTime: 650276175044 - TotalNetworkSendTime: 1576055 - TotalStorageWaitTime: 0 - TotalTime: 684551978679 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4718608 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 1093903927 - CompileTime: 1468465272 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 106410204004 - PrepareTime: 31539617610 - TotalTime: 140380860755 DataStreamSender (dst_id=25) - BytesSent: 1900 - InactiveTotalTime: 0 - NetworkThroughput(*): 1286250 - OverallThroughput: 10550631 - PeakMemoryUsage: 4096 - SerializeBatchTime: 103840 - ThriftTransmitTime(*): 1477162 - TotalTime: 180084 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 684231005214 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2696294 - GetNewBlockTime: 426976911 - GetResultsTime: 443204 - HTResizeTime: 939503 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2691768 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 1011 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 684225457818 - UnpinTime: 21018 EXCHANGE_NODE (id=23) - BytesReceived: 67579 - ConvertRowBatchTime: 764882 - DeserializeRowBatchTimer: 1220518 - FirstBatchArrivalWaitTime: 605492729121 - InactiveTotalTime: 650276028915 - PeakMemoryUsage: 0 - RowsReturned: 3673 - RowsReturnedRate: 5 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 650277041526 Instance 6643d7f56f3d4163:bb041e84a766fd97 (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) - AverageThreadTokens: 1.0 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2737000 - PerHostPeakMemUsage: 86460952 - PrepareTime: 49589053721 - RowsProduced: 100 - TotalCpuTime: 178230690995 - TotalNetworkReceiveTime: 614391051194 - TotalNetworkSendTime: 343968 - TotalStorageWaitTime: 0 - TotalTime: 665127453860 BlockMgr - BlockWritesOutstanding: 0 - BlocksCreated: 97 - BlocksRecycled: 31 - BufferedPins: 0 - BytesWritten: 0 - InactiveTotalTime: 0 - MaxBlockSize: 8388608 - MemoryLimit: 431174464 - PeakMemoryUsage: 4997136 - TotalBufferWaitTime: 0 - TotalEncryptionTime: 0 - TotalIntegrityCheckTime: 0 - TotalReadBlockTime: 0 - TotalTime: 0 CodeGen - CodegenTime: 186037994 - CompileTime: 4084189331 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 121851616110 - PrepareTime: 47081810501 - TotalTime: 173934779822 DataStreamSender (dst_id=25) - BytesSent: 1836 - InactiveTotalTime: 0 - NetworkThroughput(*): 4796764 - OverallThroughput: 7237179 - PeakMemoryUsage: 4096 - SerializeBatchTime: 83241 - ThriftTransmitTime(*): 382758 - TotalTime: 253690 - UncompressedRowBatchSize: 5404 SORT_NODE (id=20) ExecOption: Codegen Enabled - InactiveTotalTime: 0 - PeakMemoryUsage: 20480 - RowsReturned: 100 - RowsReturnedRate: 0 - TotalTime: 663990634325 AGGREGATION_NODE (id=24) ExecOption: Codegen Enabled - BuildTime: 2770818 - GetNewBlockTime: 713691094 - GetResultsTime: 454144 - HTResizeTime: 998643 - HashBuckets: 32768 - InactiveTotalTime: 0 - LargestPartitionPercent: 2 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 2690920 - PinTime: 0 - RowsRepartitioned: 0 - RowsReturned: 990 - RowsReturnedRate: 1 - SpilledPartitions: 0 - TotalTime: 663984439637 - UnpinTime: 19203 EXCHANGE_NODE (id=23) - BytesReceived: 67276 - ConvertRowBatchTime: 766089 - DeserializeRowBatchTimer: 1092515 - FirstBatchArrivalWaitTime: 569622872342 - InactiveTotalTime: 614390911541 - PeakMemoryUsage: 0 - RowsReturned: 3635 - RowsReturnedRate: 5 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 614391905799 Fragment F00 Instance 6643d7f56f3d4163:bb041e84a766fd9e (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:16/1.44 GB - AverageThreadTokens: 1.473718364698248 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 74648072 - PerHostPeakMemUsage: 77031944 - PrepareTime: 33972670968 - RowsProduced: 3685 - TotalCpuTime: 848726251463 - TotalNetworkReceiveTime: 289467344907 - TotalNetworkSendTime: 7704034 - TotalStorageWaitTime: 2367707753 - TotalTime: 696731062500 CodeGen - CodegenTime: 779181682 - CompileTime: 3653004917 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 74358363288 - PrepareTime: 30206424770 - TotalTime: 108776831447 DataStreamSender (dst_id=23) - BytesSent: 67960 - InactiveTotalTime: 0 - NetworkThroughput(*): 4584545 - OverallThroughput: 11092082 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2484633 - ThriftTransmitTime(*): 14823717 - TotalTime: 6126893 - UncompressedRowBatchSize: 199022 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 142055335 - GetResultsTime: 1062996 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25982208 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3685 - RowsReturnedRate: 5 - StreamingTime: 1023748641 - TotalTime: 695084778552 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24671488 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 662357054359 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 5786293860 - PeakMemoryUsage: 49160 - ProbeRows: 648271 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 7436828452 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6180682680716165 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 179176 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1650534592 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 7245 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 10617 - GetNewBlockTime: 329939 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 1750546885 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24728848 - PinTime: 0 - ProbeRows: 4480754 - ProbeRowsPartitioned: 0 - ProbeTime: 633517490 - RowsReturned: 179176 - RowsReturnedRate: 273 - SpilledPartitions: 0 - TotalTime: 654664279936 - UnpinTime: 18535 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3421 - DeserializeRowBatchTimer: 27187 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 36897 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 27102 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 4271244489 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 464403246 - GetNewBlockTime: 160925 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 13834762646 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 38286592 - PinTime: 0 - ProbeRows: 4480754 - ProbeRowsPartitioned: 0 - ProbeTime: 6143744074 - RowsReturned: 4480754 - RowsReturnedRate: 6862 - SpilledPartitions: 0 - TotalTime: 652913705949 - UnpinTime: 16570 EXCHANGE_NODE (id=21) - BytesReceived: 2358602 - ConvertRowBatchTime: 698144382 - DeserializeRowBatchTimer: 3500424196 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 289466410255 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 344 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 290166512720 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24638720 - RowsReturned: 4274176 - RowsReturnedRate: 12249 - TotalTime: 348912430583 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 13920895454 - PeakMemoryUsage: 24576 - ProbeRows: 4480754 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 212647327795 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 198726432341 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 2 - BuildTime: 9311036048 - InactiveTotalTime: 0 - LocalTime: 133142490169 - PeakMemoryUsage: 24576 - ProbeRows: 4650098 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 172124510869 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205953600603124 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1692522 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 13695145205 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 46262632591 - PeakMemoryUsage: 16384 - ProbeRows: 4650098 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 60678533381 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.747437256354718 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1692522 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14415900790 UNNEST_NODE (id=4) - AvgCollectionSize: 6.71534451154191 - InactiveTotalTime: 0 - MaxCollectionSize: 25 - MinCollectionSize: 1 - NumCollections: 252038 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2629127178 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 16 Hdfs split stats (:<# splits>/): 0:16/1.44 GB Hdfs Read Thread Concurrency Bucket: 0:99.86% 1:0.137% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:128 - AverageHdfsReadThreadConcurrency: 0.0013698630136986301 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 263790162 - BytesReadDataNodeCache: 0 - BytesReadLocal: 263482962 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 263482962 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 16 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50457072 - PerReadThreadRawHdfsThroughput: 554754589 - RemoteScanRanges: 3 - RowsRead: 49814903 - RowsReturned: 252038 - RowsReturnedRate: 1881 - ScanRangesComplete: 16 - ScannerThreadsInvoluntaryContextSwitches: 47125 - ScannerThreadsTotalWallClockTime: 365147482886 - MaterializeTupleTime(*): 359815690542 - ScannerThreadsSysTime: 1009846000 - ScannerThreadsUserTime: 100654698000 - ScannerThreadsVoluntaryContextSwitches: 355 - TotalRawHdfsReadTime(*): 475507850 - TotalReadThroughput: 355828 - TotalTime: 133934221105 Instance 6643d7f56f3d4163:bb041e84a766fd9b (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:14/1.42 GB - AverageThreadTokens: 1.502974223397224 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 84085272 - PerHostPeakMemUsage: 86460952 - PrepareTime: 58305849697 - RowsProduced: 3666 - TotalCpuTime: 957492728846 - TotalNetworkReceiveTime: 211091427079 - TotalNetworkSendTime: 8626930 - TotalStorageWaitTime: 4847687548 - TotalTime: 661228477860 CodeGen - CodegenTime: 268154615 - CompileTime: 3209845128 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 126748749228 - PrepareTime: 51220467724 - TotalTime: 182392844108 DataStreamSender (dst_id=23) - BytesSent: 67491 - InactiveTotalTime: 0 - NetworkThroughput(*): 6473732 - OverallThroughput: 10176634 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2847808 - ThriftTransmitTime(*): 10425361 - TotalTime: 6631957 - UncompressedRowBatchSize: 197996 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 207629310 - GetResultsTime: 1050395 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 25982208 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3666 - RowsReturnedRate: 5 - StreamingTime: 3311240048 - TotalTime: 659237657485 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 24671488 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 602920726240 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 6402723435 - PeakMemoryUsage: 49160 - ProbeRows: 640036 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 7911227269 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6185780918727914 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 176875 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1508503834 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 8005 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 6036 - GetNewBlockTime: 207742968 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 5316488439 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24728848 - PinTime: 0 - ProbeRows: 4413490 - ProbeRowsPartitioned: 0 - ProbeTime: 866888321 - RowsReturned: 176875 - RowsReturnedRate: 296 - SpilledPartitions: 0 - TotalTime: 595878839294 - UnpinTime: 24895 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3084 - DeserializeRowBatchTimer: 26359 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 39188 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 25518 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 5197302592 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 1012072257 - GetNewBlockTime: 292101172 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 17722382678 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 38286592 - PinTime: 0 - ProbeRows: 4413490 - ProbeRowsPartitioned: 0 - ProbeTime: 6873578694 - RowsReturned: 4413490 - RowsReturnedRate: 7473 - SpilledPartitions: 0 - TotalTime: 590562325337 - UnpinTime: 18058 EXCHANGE_NODE (id=21) - BytesReceived: 2358602 - ConvertRowBatchTime: 330736321 - DeserializeRowBatchTimer: 1330762164 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 211090470975 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 472 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 211611474039 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 24638720 - RowsReturned: 4209664 - RowsReturnedRate: 11653 - TotalTime: 361228468620 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 15385461131 - PeakMemoryUsage: 24576 - ProbeRows: 4413490 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 231410654504 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 216025193373 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 1 - BuildTime: 10656104213 - InactiveTotalTime: 0 - LocalTime: 146557113339 - PeakMemoryUsage: 24576 - ProbeRows: 4579880 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 186705326439 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205449388297857 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667233 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14804572291 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 49711633588 - PeakMemoryUsage: 16384 - ProbeRows: 4579880 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 64160406979 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.746994571244691 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1667233 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14448773391 UNNEST_NODE (id=4) - AvgCollectionSize: 6.713239379907389 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248350 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2933796124 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 14 Hdfs split stats (:<# splits>/): 0:14/1.42 GB Hdfs Read Thread Concurrency Bucket: 0:99.87% 1:0.1312% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:112 - AverageHdfsReadThreadConcurrency: 0.0013123359580052493 - AverageScannerThreadConcurrency: 0.9986876640419947 - BytesRead: 255461223 - BytesReadDataNodeCache: 0 - BytesReadLocal: 255256423 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 255256423 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 14 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 59894272 - PerReadThreadRawHdfsThroughput: 436529903 - RemoteScanRanges: 2 - RowsRead: 49101937 - RowsReturned: 248350 - RowsReturnedRate: 1958 - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 43982 - ScannerThreadsTotalWallClockTime: 381129814149 - MaterializeTupleTime(*): 374454476885 - ScannerThreadsSysTime: 416936000 - ScannerThreadsUserTime: 101721535000 - ScannerThreadsVoluntaryContextSwitches: 432 - TotalRawHdfsReadTime(*): 585208988 - TotalReadThroughput: 360773 - TotalTime: 126806174605 Instance 6643d7f56f3d4163:bb041e84a766fd9c (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:17/1.41 GB - AverageThreadTokens: 1.4807562457798784 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 74648080 - PerHostPeakMemUsage: 77023760 - PrepareTime: 82570285818 - RowsProduced: 3645 - TotalCpuTime: 920765481191 - TotalNetworkReceiveTime: 196292079082 - TotalNetworkSendTime: 18288475 - TotalStorageWaitTime: 5287129873 - TotalTime: 647598244304 CodeGen - CodegenTime: 1712270181 - CompileTime: 6177799871 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 111694920393 - PrepareTime: 73604458995 - TotalTime: 193026906161 DataStreamSender (dst_id=23) - BytesSent: 67145 - InactiveTotalTime: 0 - NetworkThroughput(*): 2058223 - OverallThroughput: 11568428 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2921508 - ThriftTransmitTime(*): 32622792 - TotalTime: 5804159 - UncompressedRowBatchSize: 196862 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 531361077 - GetResultsTime: 1045022 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24933632 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3645 - RowsReturnedRate: 5 - StreamingTime: 3626829411 - TotalTime: 644053394234 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 23622912 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 562730079534 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 6668133674 - PeakMemoryUsage: 49160 - ProbeRows: 632652 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 8139045452 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.612491363509185 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 175129 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1470911778 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 7333 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 6188 - GetNewBlockTime: 705354118 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 5899625970 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 23680272 - PinTime: 0 - ProbeRows: 4370971 - ProbeRowsPartitioned: 0 - ProbeTime: 1905222980 - RowsReturned: 175129 - RowsReturnedRate: 315 - SpilledPartitions: 0 - TotalTime: 555332603338 - UnpinTime: 20454 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 3148 - DeserializeRowBatchTimer: 14845 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 37098 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 26955 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 6216995543 - BuildRows: 98976 - BuildRowsPartitioned: 98976 - BuildTime: 436706753 - GetNewBlockTime: 182494 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 16299462848 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37238016 - PinTime: 0 - ProbeRows: 4416256 - ProbeRowsPartitioned: 0 - ProbeTime: 5816143460 - RowsReturned: 4370971 - RowsReturnedRate: 7955 - SpilledPartitions: 0 - TotalTime: 549432950413 - UnpinTime: 16805 EXCHANGE_NODE (id=21) - BytesReceived: 2334201 - ConvertRowBatchTime: 138462176 - DeserializeRowBatchTimer: 14267738124 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 196291085678 - PeakMemoryUsage: 0 - RowsReturned: 98976 - RowsReturnedRate: 502 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 196792626523 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 23590144 - RowsReturned: 4213760 - RowsReturnedRate: 12528 - TotalTime: 336340861042 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 14675260090 - PeakMemoryUsage: 24576 - ProbeRows: 4416256 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 215643458040 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 200968197950 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 5 - BuildTime: 9202810632 - InactiveTotalTime: 0 - LocalTime: 135366865806 - PeakMemoryUsage: 24576 - ProbeRows: 4583146 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 174021929517 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205374756109392 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1669806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 14615056548 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 47622776204 - PeakMemoryUsage: 16384 - ProbeRows: 4583146 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 60824051681 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.7447176498347714 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1669806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 13201275477 UNNEST_NODE (id=4) - AvgCollectionSize: 6.711277059234906 - InactiveTotalTime: 0 - MaxCollectionSize: 24 - MinCollectionSize: 1 - NumCollections: 248806 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2424101788 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 17 Hdfs split stats (:<# splits>/): 0:17/1.41 GB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:136 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 263557085 - BytesReadDataNodeCache: 0 - BytesReadLocal: 263454685 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 263454685 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 17 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 50457080 - PerReadThreadRawHdfsThroughput: 482031868 - RemoteScanRanges: 1 - RowsRead: 49173693 - RowsReturned: 248806 - RowsReturnedRate: 2112 - ScanRangesComplete: 17 - ScannerThreadsInvoluntaryContextSwitches: 47965 - ScannerThreadsTotalWallClockTime: 356167035396 - MaterializeTupleTime(*): 349282465262 - ScannerThreadsSysTime: 400939000 - ScannerThreadsUserTime: 103756226000 - ScannerThreadsVoluntaryContextSwitches: 449 - TotalRawHdfsReadTime(*): 546762781 - TotalReadThroughput: 383774 - TotalTime: 117753506178 Instance 6643d7f56f3d4163:bb041e84a766fd9d (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:14/1.41 GB - AverageThreadTokens: 1.4546065904505716 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 87053592 - PerHostPeakMemUsage: 89429272 - PrepareTime: 38246996803 - RowsProduced: 3632 - TotalCpuTime: 846345324646 - TotalNetworkReceiveTime: 236988336897 - TotalNetworkSendTime: 10170276 - TotalStorageWaitTime: 2518678463 - TotalTime: 622729800485 CodeGen - CodegenTime: 100791517 - CompileTime: 4436943306 - InactiveTotalTime: 0 - LoadTime: 0 - ModuleBitcodeSize: 1939944 - OptimizationTime: 120290293369 - PrepareTime: 34734916663 - TotalTime: 159877007072 DataStreamSender (dst_id=23) - BytesSent: 66929 - InactiveTotalTime: 0 - NetworkThroughput(*): 3106364 - OverallThroughput: 12935609 - PeakMemoryUsage: 163840 - SerializeBatchTime: 2480606 - ThriftTransmitTime(*): 21545765 - TotalTime: 5174012 - UncompressedRowBatchSize: 196160 AGGREGATION_NODE (id=19) ExecOption: Streaming Preaggregation, Codegen Enabled - GetNewBlockTime: 204400 - GetResultsTime: 1133287 - HTResizeTime: 0 - HashBuckets: 16384 - InactiveTotalTime: 0 - LargestPartitionPercent: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 24933632 - PinTime: 0 - ReductionFactorEstimate: 0.0 - ReductionFactorThresholdToExpand: 0.0 - RowsPassedThrough: 0 - RowsReturned: 3632 - RowsReturnedRate: 5 - StreamingTime: 3013315310 - TotalTime: 620631142565 - UnpinTime: 0 SUBPLAN_NODE (id=18) - InactiveTotalTime: 0 - PeakMemoryUsage: 23622912 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 582303467327 NESTED_LOOP_JOIN_NODE (id=16) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 6744613255 - PeakMemoryUsage: 49160 - ProbeRows: 629454 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 8447787237 SINGULAR_ROW_SRC_NODE (id=14) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=15) - AvgCollectionSize: 3.6116982820945362 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 174282 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 1703173982 HASH_JOIN_NODE (id=17) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 9051 - BuildRows: 1 - BuildRowsPartitioned: 1 - BuildTime: 9406 - GetNewBlockTime: 843618278 - HashBuckets: 1 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 100 - LocalTime: 3935758387 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 23680272 - PinTime: 0 - ProbeRows: 4343863 - ProbeRowsPartitioned: 0 - ProbeTime: 746310528 - RowsReturned: 174282 - RowsReturnedRate: 304 - SpilledPartitions: 0 - TotalTime: 573063139860 - UnpinTime: 22023 EXCHANGE_NODE (id=22) - BytesReceived: 41 - ConvertRowBatchTime: 4009 - DeserializeRowBatchTimer: 15377 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 1 - RowsReturnedRate: 33896 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 29502 HASH_JOIN_NODE (id=13) ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled - BuildPartitionTime: 4290238213 - BuildRows: 100000 - BuildRowsPartitioned: 100000 - BuildTime: 381971129 - GetNewBlockTime: 222580 - HashBuckets: 262144 - HashCollisions: 0 - InactiveTotalTime: 0 - LargestPartitionPercent: 6 - LocalTime: 13473272173 - MaxPartitionLevel: 0 - NumRepartitions: 0 - PartitionsCreated: 16 - PeakMemoryUsage: 37238016 - PinTime: 0 - ProbeRows: 4343863 - ProbeRowsPartitioned: 0 - ProbeTime: 7036925521 - RowsReturned: 4343863 - RowsReturnedRate: 7632 - SpilledPartitions: 0 - TotalTime: 569127351971 - UnpinTime: 17587 EXCHANGE_NODE (id=21) - BytesReceived: 2358602 - ConvertRowBatchTime: 1279914638 - DeserializeRowBatchTimer: 5981081993 - FirstBatchArrivalWaitTime: 0 - InactiveTotalTime: 236987296081 - PeakMemoryUsage: 0 - RowsReturned: 100000 - RowsReturnedRate: 419 - SendersBlockedTimer: 0 - SendersBlockedTotalTimer(*): 0 - TotalTime: 238404169203 SUBPLAN_NODE (id=2) - InactiveTotalTime: 0 - PeakMemoryUsage: 23590144 - RowsReturned: 4145152 - RowsReturnedRate: 13065 - TotalTime: 317249910595 NESTED_LOOP_JOIN_NODE (id=11) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 16181328663 - PeakMemoryUsage: 24576 - ProbeRows: 4343863 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 236873925245 SINGULAR_ROW_SRC_NODE (id=3) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 SUBPLAN_NODE (id=5) - InactiveTotalTime: 0 - PeakMemoryUsage: 8192 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 220692596582 NESTED_LOOP_JOIN_NODE (id=10) - BuildRows: 2 - BuildTime: 10345925450 - InactiveTotalTime: 0 - LocalTime: 147263933643 - PeakMemoryUsage: 24576 - ProbeRows: 4508276 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 190640792140 UNNEST_NODE (id=8) - AvgCollectionSize: 4.205691209456636 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1641514 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 16054505553 NESTED_LOOP_JOIN_NODE (id=9) - BuildRows: 0 - BuildTime: 0 - InactiveTotalTime: 0 - LocalTime: 51888231852 - PeakMemoryUsage: 16384 - ProbeRows: 4508276 - ProbeTime: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 67420045965 SINGULAR_ROW_SRC_NODE (id=6) - InactiveTotalTime: 0 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 0 UNNEST_NODE (id=7) - AvgCollectionSize: 2.746413372045563 - InactiveTotalTime: 0 - MaxCollectionSize: 7 - MinCollectionSize: 1 - NumCollections: 1641514 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 15531814113 UNNEST_NODE (id=4) - AvgCollectionSize: 6.698744735725246 - InactiveTotalTime: 0 - MaxCollectionSize: 25 - MinCollectionSize: 1 - NumCollections: 245048 - PeakMemoryUsage: 0 - RowsReturned: 0 - RowsReturnedRate: 0 - TotalTime: 2642323867 HDFS_SCAN_NODE (id=1) ExecOption: Expr Evaluation Codegen Enabled, Codegen enabled: 0 out of 14 Hdfs split stats (:<# splits>/): 0:14/1.41 GB Hdfs Read Thread Concurrency Bucket: 0:99.85% 1:0.1479% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:112 - AverageHdfsReadThreadConcurrency: 0.0014792899408284023 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 252141413 - BytesReadDataNodeCache: 0 - BytesReadLocal: 251834213 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 251834213 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 8 - NumDisksAccessed: 2 - NumRowGroups: 14 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 62862592 - PerReadThreadRawHdfsThroughput: 387891459 - RemoteScanRanges: 3 - RowsRead: 48367623 - RowsReturned: 245048 - RowsReturnedRate: 3159 - ScanRangesComplete: 14 - ScannerThreadsInvoluntaryContextSwitches: 47593 - ScannerThreadsTotalWallClockTime: 337983987920 - MaterializeTupleTime(*): 334061887243 - ScannerThreadsSysTime: 406938000 - ScannerThreadsUserTime: 102038487000 - ScannerThreadsVoluntaryContextSwitches: 346 - TotalRawHdfsReadTime(*): 650030844 - TotalReadThroughput: 355566 - TotalTime: 77555801125 Fragment F02 Instance 6643d7f56f3d4163:bb041e84a766fd9f (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:1/4.17 KB - AverageThreadTokens: 1.2384105960264902 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 73024 - PerHostPeakMemUsage: 7302872 - PrepareTime: 375623 - RowsProduced: 1 - TotalCpuTime: 23275256541 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 58031767132 - TotalStorageWaitTime: 14813344730 - TotalTime: 19896548298 DataStreamSender (dst_id=22) - BytesSent: 164 - InactiveTotalTime: 0 - NetworkThroughput(*): 1 - OverallThroughput: 592 - PeakMemoryUsage: 21824 - SerializeBatchTime: 276540763 - ThriftTransmitTime(*): 123320861267 - TotalTime: 276667567 - UncompressedRowBatchSize: 176 HDFS_SCAN_NODE (id=12) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 1 Hdfs split stats (:<# splits>/): 0:1/4.17 KB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:2 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 1.0 - BytesRead: 4758 - BytesReadDataNodeCache: 0 - BytesReadLocal: 4758 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 4758 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 2 - NumDisksAccessed: 1 - NumRowGroups: 1 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 43008 - PerReadThreadRawHdfsThroughput: 130750206 - RemoteScanRanges: 0 - RowsRead: 25 - RowsReturned: 1 - RowsReturnedRate: 0 - ScanRangesComplete: 1 - ScannerThreadsInvoluntaryContextSwitches: 0 - ScannerThreadsTotalWallClockTime: 18082768188 - MaterializeTupleTime(*): 77953 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 0 - ScannerThreadsVoluntaryContextSwitches: 21 - TotalRawHdfsReadTime(*): 36390 - TotalReadThroughput: 226 - TotalTime: 18348243113 Fragment F01 Instance 6643d7f56f3d4163:bb041e84a766fda0 (host=impala-stress-cdh570-3.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:3/278.37 MB - AverageThreadTokens: 1.7697456492637216 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1034240 - PerHostPeakMemUsage: 18715528 - PrepareTime: 279059 - RowsProduced: 25022 - TotalCpuTime: 321805266953 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 126915315992 - TotalStorageWaitTime: 319686124709 - TotalTime: 407489926955 DataStreamSender (dst_id=21) - BytesSent: 2366228 - InactiveTotalTime: 0 - NetworkThroughput(*): 18053 - OverallThroughput: 18640 - PeakMemoryUsage: 16384 - SerializeBatchTime: 46495796 - ThriftTransmitTime(*): 131070625749 - TotalTime: 126941491864 - UncompressedRowBatchSize: 5405168 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 3 Hdfs split stats (:<# splits>/): 0:3/278.37 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:9 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.777027027027027 - BytesRead: 1074417 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1074417 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1074417 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 3 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 976896 - PerReadThreadRawHdfsThroughput: 217062358 - RemoteScanRanges: 0 - RowsRead: 25022 - RowsReturned: 25022 - RowsReturnedRate: 89 - ScanRangesComplete: 3 - ScannerThreadsInvoluntaryContextSwitches: 43 - ScannerThreadsTotalWallClockTime: 321805276943 - MaterializeTupleTime(*): 1864721685 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 28995000 - ScannerThreadsVoluntaryContextSwitches: 43 - TotalRawHdfsReadTime(*): 4949808 - TotalReadThroughput: 2871 - TotalTime: 278352908282 Instance 6643d7f56f3d4163:bb041e84a766fda2 (host=impala-stress-cdh570-4.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:2/256.00 MB - AverageThreadTokens: 1.784037558685446 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 1167360 - PerHostPeakMemUsage: 9165992 - PrepareTime: 63030432 - RowsProduced: 23012 - TotalCpuTime: 106077676009 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 42813952753 - TotalStorageWaitTime: 91024346657 - TotalTime: 111605981423 DataStreamSender (dst_id=21) - BytesSent: 2172492 - InactiveTotalTime: 0 - NetworkThroughput(*): 36504 - OverallThroughput: 50709 - PeakMemoryUsage: 16384 - SerializeBatchTime: 17704075 - ThriftTransmitTime(*): 59512224303 - TotalTime: 42841892661 - UncompressedRowBatchSize: 4970976 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 2 Hdfs split stats (:<# splits>/): 0:2/256.00 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:6 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.8789473684210526 - BytesRead: 910178 - BytesReadDataNodeCache: 0 - BytesReadLocal: 910178 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 910178 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 2 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 1110016 - PerReadThreadRawHdfsThroughput: 249630016 - RemoteScanRanges: 0 - RowsRead: 23012 - RowsReturned: 23012 - RowsReturnedRate: 342 - ScanRangesComplete: 2 - ScannerThreadsInvoluntaryContextSwitches: 17 - ScannerThreadsTotalWallClockTime: 106077685017 - MaterializeTupleTime(*): 14517484424 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 28995000 - ScannerThreadsVoluntaryContextSwitches: 119 - TotalRawHdfsReadTime(*): 3646108 - TotalReadThroughput: 9353 - TotalTime: 67201907210 Instance 6643d7f56f3d4163:bb041e84a766fda1 (host=impala-stress-cdh570-2.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:4/303.37 MB - AverageThreadTokens: 1.3558282208588956 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2340864 - PerHostPeakMemUsage: 8144512 - PrepareTime: 103681768 - RowsProduced: 27271 - TotalCpuTime: 30444860144 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 74520193742 - TotalStorageWaitTime: 27728104545 - TotalTime: 82065530539 DataStreamSender (dst_id=21) - BytesSent: 2605576 - InactiveTotalTime: 0 - NetworkThroughput(*): 14928 - OverallThroughput: 34831 - PeakMemoryUsage: 16384 - SerializeBatchTime: 265947617 - ThriftTransmitTime(*): 174535578050 - TotalTime: 74806127724 - UncompressedRowBatchSize: 5891016 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 4 Hdfs split stats (:<# splits>/): 0:4/303.37 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.36942675159235666 - BytesRead: 1245972 - BytesReadDataNodeCache: 0 - BytesReadLocal: 1245972 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 1245972 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 4 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 2283520 - PerReadThreadRawHdfsThroughput: 3298312 - RemoteScanRanges: 0 - RowsRead: 27271 - RowsReturned: 27271 - RowsReturnedRate: 4673 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 44 - ScannerThreadsTotalWallClockTime: 30444867642 - MaterializeTupleTime(*): 1604682747 - ScannerThreadsSysTime: 999000 - ScannerThreadsUserTime: 32994000 - ScannerThreadsVoluntaryContextSwitches: 97 - TotalRawHdfsReadTime(*): 377760454 - TotalReadThroughput: 15464 - TotalTime: 5835292812 Instance 6643d7f56f3d4163:bb041e84a766fda3 (host=impala-stress-cdh570-5.vpc.cloudera.com:22000) Hdfs split stats (:<# splits>/): 0:4/274.81 MB - AverageThreadTokens: 1.955128205128205 - BloomFilterBytes: 0 - InactiveTotalTime: 0 - PeakMemoryUsage: 2054144 - PerHostPeakMemUsage: 7697248 - PrepareTime: 206488 - RowsProduced: 24695 - TotalCpuTime: 76631928993 - TotalNetworkReceiveTime: 0 - TotalNetworkSendTime: 63706248111 - TotalStorageWaitTime: 74193417023 - TotalTime: 79739539558 DataStreamSender (dst_id=21) - BytesSent: 2290112 - InactiveTotalTime: 0 - NetworkThroughput(*): 16708 - OverallThroughput: 35668 - PeakMemoryUsage: 16384 - SerializeBatchTime: 470205414 - ThriftTransmitTime(*): 137066103428 - TotalTime: 64206070100 - UncompressedRowBatchSize: 5334552 HDFS_SCAN_NODE (id=0) ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 4 Hdfs split stats (:<# splits>/): 0:4/274.81 MB Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% File Formats: PARQUET/NONE:12 - AverageHdfsReadThreadConcurrency: 0.0 - AverageScannerThreadConcurrency: 0.9802631578947368 - BytesRead: 1167078 - BytesReadDataNodeCache: 0 - BytesReadLocal: 962278 - BytesReadRemoteUnexpected: 0 - BytesReadShortCircuit: 962278 - DecompressionTime: 0 - InactiveTotalTime: 0 - MaxCompressedTextFileLength: 0 - NumColumns: 3 - NumDisksAccessed: 2 - NumRowGroups: 4 - NumScannerThreadsStarted: 1 - PeakMemoryUsage: 1996800 - PerReadThreadRawHdfsThroughput: 6276957 - RemoteScanRanges: 2 - RowsRead: 24695 - RowsReturned: 24695 - RowsReturnedRate: 1777 - ScanRangesComplete: 4 - ScannerThreadsInvoluntaryContextSwitches: 28 - ScannerThreadsTotalWallClockTime: 76631934536 - MaterializeTupleTime(*): 849244361 - ScannerThreadsSysTime: 0 - ScannerThreadsUserTime: 31995000 - ScannerThreadsVoluntaryContextSwitches: 92 - TotalRawHdfsReadTime(*): 185930534 - TotalReadThroughput: 14950 - TotalTime: 13892616496