[DRILL-6897] TPCH 13 has regressed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.15.0
• Component/s: Execution - Relational Operators
• Assignee: Karthikeyan Manivannan
• Reporter: Robert Hou
• Created: 12/Dec/18 01:42
• Updated: 11/Jan/19 19:51
• Resolved: 11/Jan/19 19:51

Description

I ran TPCH query 13 with both scale factor 100 and 1000, and ran them 3x to get a warm start, and ran them twice to verify the regression. It is regressing between 26 and 33%.

Here is the query:

select
  c_count,
  count(*) as custdist
from
  (
    select
      c.c_custkey,
      count(o.o_orderkey)
    from
      customer c 
      left outer join orders o 
        on c.c_custkey = o.o_custkey
        and o.o_comment not like '%special%requests%'
    group by
      c.c_custkey
  ) as orders (c_custkey, c_count)
group by
  c_count
order by
  custdist desc,
  c_count desc;

I have attached two profiles. 240099ed-ef2a-a23a-4559-f1b2e0809e72 is for Drill 1.15. 2400be84-c024-cb92-8743-3211589e0247 is for Drill 1.14. The commit for Drill 1.15 is 596227bbbecfb19bdb55dd8ea58159890f83bc9c. The commit for Drill 1.14 is 0508a128853ce796ca7e99e13008e49442f83147.

The two plans nearly the same. One difference is that Drill 1.15 is using four times more memory in operator 07-01 Unordered Mux Exchange. I think the problem may be in operator 09-01 Project. Drill 1.15 is projecting the comment field while Drill 1.14 does not project the comment field.

Another issue is that the Drill 1.15 takes more processing time to filter the order table. Filter operator 09-03 takes an average of 19.3s. For Drill 1.14, filter operator 09-04 takes an average of 15.6s. They process the same number of rows, and have the same number of minor fragments.

Issue Links

• duplicates: DRILL-6896 Extraneous columns being projected past a join