Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.8.0
Description
With mt_dop > 0 queries skip admission control
Query (id=c34b4445491d7984:4514bfc000000000):
Summary:
Session ID: 7743a91e02573fae:a8f116736e980bad
Session Type: BEESWAX
Start Time: 2016-12-09 14:26:54.031855000
End Time: 2016-12-09 14:27:00.713773000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.8.0-SNAPSHOT RELEASE (build a5176818172103693fa124d69982963b04b1098e)
User: systest@HALXG.CLOUDERA.COM
Connected User: systest@HALXG.CLOUDERA.COM
Delegated User:
Network Address: 10.17.234.19:60064
Default Db: tpch_300_parquet
Sql Statement: select count(*) from lineitem group by l_orderkey having count(*) > 10000
Coordinator: vd1309.halxg.cloudera.com:22000
Query Options (non default): NUM_SCANNER_THREADS=1,MT_DOP=8
Plan:
----------------
Estimated Per-Host Requirements: Memory=7.98GB VCores=2
WARNING: Spilling is disabled for this query as a safety guard.
Reason: Query option disable_unsafe_spills is set, at least one table
is missing relevant stats, and no plan hints were given.
PLAN-ROOT SINK
|
04:EXCHANGE [UNPARTITIONED]
| hosts=7 per-host-mem=unavailable
| tuple-ids=1 row-size=16B cardinality=43790950
|
03:AGGREGATE [FINALIZE]
| output: count:merge(*)
| group by: l_orderkey
| having: count(*) > 10000
| hosts=7 per-host-mem=735.02MB
| tuple-ids=1 row-size=16B cardinality=43790950
|
02:EXCHANGE [HASH(l_orderkey)]
| hosts=7 per-host-mem=0B
| tuple-ids=1 row-size=16B cardinality=437909504
|
01:AGGREGATE [STREAMING]
| output: count(*)
| group by: l_orderkey
| hosts=7 per-host-mem=7.18GB
| tuple-ids=1 row-size=16B cardinality=437909504
|
00:SCAN HDFS [tpch_300_parquet.lineitem, RANDOM]
partitions=1/1 files=259 size=63.71GB
table stats: 1799989091 rows total
column stats: all
hosts=7 per-host-mem=88.00MB
tuple-ids=0 row-size=8B cardinality=1799989091
----------------
Estimated Per-Host Mem: 8570202713
Estimated Per-Host VCores: 2
Request Pool: root.systest
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
----------------------------------------------------------------------------------------------------------------------
04:EXCHANGE 1 7.533us 7.533us 0 43.79M 0 -1.00 B UNPARTITIONED
03:AGGREGATE 56 2s643ms 2s861ms 0 43.79M 402.06 MB 735.02 MB FINALIZE
02:EXCHANGE 56 276.883ms 301.636ms 450.04M 437.91M 0 0 HASH(l_orderkey)
01:AGGREGATE 56 2s189ms 2s519ms 450.04M 437.91M 521.33 MB 7.18 GB STREAMING
00:SCAN HDFS 56 506.329ms 649.212ms 1.80B 1.80B 12.47 MB 88.00 MB tpch_300_parquet.lineitem
Planner Timeline: 2.319ms
- Analysis finished: 674.915us (674.915us)
- Equivalence classes computed: 766.195us (91.280us)
- Single node plan created: 1.193ms (427.359us)
- Runtime filters computed: 1.222ms (29.215us)
- Distributed plan created: 1.372ms (149.684us)
- Lineage info computed: 1.413ms (41.209us)
- Parallel plans created: 1.465ms (51.646us)
- Planning finished: 2.319ms (854.253us)
Query Timeline: 6s705ms
- Query submitted: 70.897us (70.897us)
- Planning finished: 3.788ms (3.717ms)
- Ready to start 113 fragment instances: 5.457ms (1.669ms)
- All 113 fragment instances started: 157.998ms (152.540ms)
- Rows available: 6s539ms (6s381ms)
- First row fetched: 6s627ms (87.972ms)
- Unregister query: 6s681ms (54.351ms)
- ComputeScanRangeAssignmentTimer: 445.101us