Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-560

CTAS PARTITION BY with UNION can cause invalid global plan

    Details

      Description

      Take a look at the following query. This kind of queries can cause invalid distributed plans.

      create table testCtasWithUnion (col1 int4, col2 int4) partition by column(key float8) as
      
      select
        *
      from (
      
      select
        sum(l_orderkey) as total1,
        avg(l_partkey) as total2,
        l_quantity as key
      from
        lineitem
      group by
        l_quantity
      order by
        l_quantity
      limit
        3
      
      union
      
      select
        sum(l_orderkey) as total1,
        avg(l_partkey) as total2,
        l_quantity as key
      from
        lineitem
      group by
        l_quantity
      order by
        l_quantity
      limit
        3
      
      ) t1;
      
      1. TAJO-560_2.patch
        33 kB
        Hyunsik Choi
      2. TAJO-560.patch
        33 kB
        Hyunsik Choi

        Activity

        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-build #37 (See https://builds.apache.org/job/Tajo-master-build/37/)
        TAJO-560: CTAS PARTITION BY with UNION can cause invalid global plan. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=6aa96fa9bc58d002efeae03f775162ccee6545e5)

        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithGroupby.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/check1.sql
        • tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithLimit.result
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk1.sql
        • CHANGES.txt
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk2.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithColumnedPartition.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithLimit.sql
        • tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithUnion.result
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithGroupby.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/check2.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithOrderby.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithUnion.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk3.sql
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithLimit.sql
        • tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterik.result
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithoutTableDefinition.sql
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithOrderby.sql
        • tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk2.result
        • tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithUnion.sql
        • tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk3.result
        • tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithOrderby.result
        • tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk1.result
        • tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithGroupby.result
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-build #37 (See https://builds.apache.org/job/Tajo-master-build/37/ ) TAJO-560 : CTAS PARTITION BY with UNION can cause invalid global plan. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=6aa96fa9bc58d002efeae03f775162ccee6545e5 ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithGroupby.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/check1.sql tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithLimit.result tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk1.sql CHANGES.txt tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk2.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithColumnedPartition.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithLimit.sql tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithUnion.result tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithGroupby.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/check2.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithOrderby.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithUnion.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestSelectQuery/testSelectAsterisk3.sql tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithLimit.sql tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterik.result tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithoutTableDefinition.sql tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testCtasWithOrderby.sql tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk2.result tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/CtasWithUnion.sql tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk3.result tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithOrderby.result tajo-core/tajo-core-backend/src/test/resources/results/TestSelectQuery/testSelectAsterisk1.result tajo-core/tajo-core-backend/src/test/resources/results/TestCTASQuery/testCtasWithGroupby.result
        Hide
        hyunsik Hyunsik Choi added a comment -

        It got +1 in RB. I've committed it to master branch.

        Show
        hyunsik Hyunsik Choi added a comment - It got +1 in RB. I've committed it to master branch.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I uploaded the new patch. The unit test failures are caused by different result orders of some operators. I added order-by clauses to the failed unit tests.

        Show
        hyunsik Hyunsik Choi added a comment - I uploaded the new patch. The unit test failures are caused by different result orders of some operators. I added order-by clauses to the failed unit tests.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I fixed GlobalPlanner and refactored buildStorePlan(). I verified 'mvn clean install'.

        Show
        hyunsik Hyunsik Choi added a comment - I fixed GlobalPlanner and refactored buildStorePlan(). I verified 'mvn clean install'.
        Hide
        hyunsik Hyunsik Choi added a comment -

        Created a review request against branch master in reviewboard
        https://reviews.apache.org/r/17448/

        Show
        hyunsik Hyunsik Choi added a comment - Created a review request against branch master in reviewboard https://reviews.apache.org/r/17448/

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development