Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6897

TPCH 13 has regressed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.15.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      I ran TPCH query 13 with both scale factor 100 and 1000, and ran them 3x to get a warm start, and ran them twice to verify the regression. It is regressing between 26 and 33%.

      Here is the query:

      select
        c_count,
        count(*) as custdist
      from
        (
          select
            c.c_custkey,
            count(o.o_orderkey)
          from
            customer c 
            left outer join orders o 
              on c.c_custkey = o.o_custkey
              and o.o_comment not like '%special%requests%'
          group by
            c.c_custkey
        ) as orders (c_custkey, c_count)
      group by
        c_count
      order by
        custdist desc,
        c_count desc;
      

      I have attached two profiles. 240099ed-ef2a-a23a-4559-f1b2e0809e72 is for Drill 1.15. 2400be84-c024-cb92-8743-3211589e0247 is for Drill 1.14. The commit for Drill 1.15 is 596227bbbecfb19bdb55dd8ea58159890f83bc9c. The commit for Drill 1.14 is 0508a128853ce796ca7e99e13008e49442f83147.

      The two plans nearly the same. One difference is that Drill 1.15 is using four times more memory in operator 07-01 Unordered Mux Exchange. I think the problem may be in operator 09-01 Project. Drill 1.15 is projecting the comment field while Drill 1.14 does not project the comment field.

      Another issue is that the Drill 1.15 takes more processing time to filter the order table. Filter operator 09-03 takes an average of 19.3s. For Drill 1.14, filter operator 09-04 takes an average of 15.6s. They process the same number of rows, and have the same number of minor fragments.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                karthikm Karthikeyan Manivannan
                Reporter:
                rhou Robert Hou
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: