[IMPALA-3087] Explain plan regression due to codegen of disjunct predicates

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.5.0
• Component/s: Frontend
• Labels: performance
• Target Version: Impala 2.5.0
• Assignee: Michael Ho
• Reporter: Mostafa Mokhtar
• Created: 25/Feb/16 23:06
• Updated: 26/Feb/16 18:36
• Resolved: 26/Feb/16 18:36

Description

TPC-DS Q3 has regressed in 2.5 compared to 2.3 due to query planning taking 6s compared to 1.3s for 2.3

Query

select
  dt.d_year,
  item.i_brand_id brand_id,
  item.i_brand brand,
  sum(ss_ext_sales_price) sum_agg
from
  date_dim dt,
  store_sales,
  item
where
  dt.d_date_sk = store_sales.ss_sold_date_sk
  and store_sales.ss_item_sk = item.i_item_sk
  and item.i_manufact_id = 436
  and dt.d_moy = 12
  -- partition key filters
  and (ss_sold_date_sk between 2451149 and 2451179
    or ss_sold_date_sk between 2451514 and 2451544
    or ss_sold_date_sk between 2451880 and 2451910
    or ss_sold_date_sk between 2452245 and 2452275
    or ss_sold_date_sk between 2452610 and 2452640)
group by
  dt.d_year,
  item.i_brand,
  item.i_brand_id
order by
  dt.d_year,
  sum_agg desc,
  brand_id
limit 100

From the profile

    Planner Timeline: 6s103ms
       - Analysis finished: 10.923ms (10.923ms)
       - Equivalence classes computed: 11.240ms (316.228us)
       - Single node plan created: 6s095ms (6s083ms)
       - Runtime filters computed: 6s095ms (142.439us)
       - Distributed plan created: 6s098ms (3.806ms)
       - Lineage info computed: 6s100ms (1.060ms)
       - Planning finished: 6s103ms (3.438ms)
    Query Timeline: 7s745ms
       - Start execution: 66.767us (66.767us)
       - Planning finished: 6s108ms (6s108ms)
       - Ready to start 32 remote fragments: 6s111ms (2.565ms)
       - All 32 remote fragments started: 6s162ms (51.360ms)
       - Rows available: 7s694ms (1s531ms)
       - First row fetched: 7s728ms (34.364ms)

During the plan generation this appears to be the hot path

"Thread-8" prio=10 tid=0x00000000068d5800 nid=0xb1c5 runnable [0x00007fbc6b667000]
   java.lang.Thread.State: RUNNABLE
        at com.cloudera.impala.service.FeSupport.NativeEvalConstExprs(Native Method)
        at com.cloudera.impala.service.FeSupport.EvalConstExprs(FeSupport.java:172)
        at com.cloudera.impala.service.FeSupport.EvalPredicateBatch(FeSupport.java:211)
        at com.cloudera.impala.planner.HdfsPartitionFilter.getMatchingPartitionIds(HdfsPartitionFilter.java:94)
        at com.cloudera.impala.planner.HdfsPartitionPruner.evalPartitionFiltersInBe(HdfsPartitionPruner.java:452)
        at com.cloudera.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:144)
        at com.cloudera.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1251)
        at com.cloudera.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1311)
        at com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1544)
        at com.cloudera.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:806)
        at com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:645)
        at com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:234)
        at com.cloudera.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:140)
        at com.cloudera.impala.planner.Planner.createPlan(Planner.java:60)
        at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:945)
        at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:141)