Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.8.0
Description
With mt_dop > 0 queries skip admission control
Query (id=c34b4445491d7984:4514bfc000000000): Summary: Session ID: 7743a91e02573fae:a8f116736e980bad Session Type: BEESWAX Start Time: 2016-12-09 14:26:54.031855000 End Time: 2016-12-09 14:27:00.713773000 Query Type: QUERY Query State: FINISHED Query Status: OK Impala Version: impalad version 2.8.0-SNAPSHOT RELEASE (build a5176818172103693fa124d69982963b04b1098e) User: systest@HALXG.CLOUDERA.COM Connected User: systest@HALXG.CLOUDERA.COM Delegated User: Network Address: 10.17.234.19:60064 Default Db: tpch_300_parquet Sql Statement: select count(*) from lineitem group by l_orderkey having count(*) > 10000 Coordinator: vd1309.halxg.cloudera.com:22000 Query Options (non default): NUM_SCANNER_THREADS=1,MT_DOP=8 Plan: ---------------- Estimated Per-Host Requirements: Memory=7.98GB VCores=2 WARNING: Spilling is disabled for this query as a safety guard. Reason: Query option disable_unsafe_spills is set, at least one table is missing relevant stats, and no plan hints were given. PLAN-ROOT SINK | 04:EXCHANGE [UNPARTITIONED] | hosts=7 per-host-mem=unavailable | tuple-ids=1 row-size=16B cardinality=43790950 | 03:AGGREGATE [FINALIZE] | output: count:merge(*) | group by: l_orderkey | having: count(*) > 10000 | hosts=7 per-host-mem=735.02MB | tuple-ids=1 row-size=16B cardinality=43790950 | 02:EXCHANGE [HASH(l_orderkey)] | hosts=7 per-host-mem=0B | tuple-ids=1 row-size=16B cardinality=437909504 | 01:AGGREGATE [STREAMING] | output: count(*) | group by: l_orderkey | hosts=7 per-host-mem=7.18GB | tuple-ids=1 row-size=16B cardinality=437909504 | 00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM] partitions=1/1 files=259 size=63.71GB table stats: 1799989091 rows total column stats: all hosts=7 per-host-mem=88.00MB tuple-ids=0 row-size=8B cardinality=1799989091 ---------------- Estimated Per-Host Mem: 8570202713 Estimated Per-Host VCores: 2 Request Pool: root.systest ExecSummary: Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail ---------------------------------------------------------------------------------------------------------------------- 04:EXCHANGE 1 7.533us 7.533us 0 43.79M 0 -1.00 B UNPARTITIONED 03:AGGREGATE 56 2s643ms 2s861ms 0 43.79M 402.06 MB 735.02 MB FINALIZE 02:EXCHANGE 56 276.883ms 301.636ms 450.04M 437.91M 0 0 HASH(l_orderkey) 01:AGGREGATE 56 2s189ms 2s519ms 450.04M 437.91M 521.33 MB 7.18 GB STREAMING 00:SCAN HDFS 56 506.329ms 649.212ms 1.80B 1.80B 12.47 MB 88.00 MB tpch_300_parquet.lineitem Planner Timeline: 2.319ms - Analysis finished: 674.915us (674.915us) - Equivalence classes computed: 766.195us (91.280us) - Single node plan created: 1.193ms (427.359us) - Runtime filters computed: 1.222ms (29.215us) - Distributed plan created: 1.372ms (149.684us) - Lineage info computed: 1.413ms (41.209us) - Parallel plans created: 1.465ms (51.646us) - Planning finished: 2.319ms (854.253us) Query Timeline: 6s705ms - Query submitted: 70.897us (70.897us) - Planning finished: 3.788ms (3.717ms) - Ready to start 113 fragment instances: 5.457ms (1.669ms) - All 113 fragment instances started: 157.998ms (152.540ms) - Rows available: 6s539ms (6s381ms) - First row fetched: 6s627ms (87.972ms) - Unregister query: 6s681ms (54.351ms) - ComputeScanRangeAssignmentTimer: 445.101us