Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-795

PlannerUtil::joinJoinKeyForEachTable need to handle theta-join.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.9.0, 0.8.1
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      Currently, Tajo supports only equi-join, thus shuffle keys have to include equi-jon conditions. But it includes theta-join conditions occasionally. We need to update this bug, because tajo may be returned wrong row numbers. Actually, I found that TPC-DS Q32 had returned unexpected results by this bug.

      1. TAJO-795_2.patch
        8 kB
        Jaehwa Jung
      2. TAJO-795_3.patch
        8 kB
        Jaehwa Jung
      3. TAJO-795.patch
        5 kB
        Jaehwa Jung

        Issue Links

          Activity

          Hide
          blrunner Jaehwa Jung added a comment -

          Created a review request against branch master in reviewboard
          https://reviews.apache.org/r/20661/

          Show
          blrunner Jaehwa Jung added a comment - Created a review request against branch master in reviewboard https://reviews.apache.org/r/20661/
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12641696/TAJO-795.patch
          against master revision 5bbb7e8.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 191 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/386//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/386//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/386//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12641696/TAJO-795.patch against master revision 5bbb7e8. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 191 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/386//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/386//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/386//console This message is automatically generated.
          Hide
          blrunner Jaehwa Jung added a comment -

          Updated the review request against branch master in reviewboard.
          https://reviews.apache.org/r/20661/

          Show
          blrunner Jaehwa Jung added a comment - Updated the review request against branch master in reviewboard. https://reviews.apache.org/r/20661/
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12642177/TAJO-795_2.patch
          against master revision 71f394d.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 191 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/388//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/388//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/388//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12642177/TAJO-795_2.patch against master revision 71f394d. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 191 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/388//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/388//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/388//console This message is automatically generated.
          Hide
          blrunner Jaehwa Jung added a comment -

          I uploaded third patch.

          Show
          blrunner Jaehwa Jung added a comment - I uploaded third patch.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12643669/TAJO-795_3.patch
          against master revision 08bcc2d.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 184 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-core.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/412//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/412//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/412//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643669/TAJO-795_3.patch against master revision 08bcc2d. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 184 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/412//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/412//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/412//console This message is automatically generated.
          Hide
          blrunner Jaehwa Jung added a comment -

          This patch got +1 at RB.
          I've just committed it to master and branch-0.8.1.

          Show
          blrunner Jaehwa Jung added a comment - This patch got +1 at RB. I've just committed it to master and branch-0.8.1.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-0.8.1-build #96 (See https://builds.apache.org/job/Tajo-0.8.1-build/96/)
          TAJO-795: PlannerUtil::joinJoinKeyForEachTable need to handle theta-join. (jaehwa) (jhjung: rev 7eb0e70106a99c157434ee8ba3c3e76f130ae275)

          • tajo-core/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
          • CHANGES
          • tajo-core/src/test/java/org/apache/tajo/master/TestGlobalPlanner.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.8.1-build #96 (See https://builds.apache.org/job/Tajo-0.8.1-build/96/ ) TAJO-795 : PlannerUtil::joinJoinKeyForEachTable need to handle theta-join. (jaehwa) (jhjung: rev 7eb0e70106a99c157434ee8ba3c3e76f130ae275) tajo-core/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java CHANGES tajo-core/src/test/java/org/apache/tajo/master/TestGlobalPlanner.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-build #213 (See https://builds.apache.org/job/Tajo-master-build/213/)
          TAJO-795: PlannerUtil::joinJoinKeyForEachTable need to handle theta-join. (jaehwa) (jhjung: rev 70567fcf47b5fdcc33a21be51c6de03e016ae4ff)

          • tajo-core/src/test/java/org/apache/tajo/master/TestGlobalPlanner.java
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
          • CHANGES
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #213 (See https://builds.apache.org/job/Tajo-master-build/213/ ) TAJO-795 : PlannerUtil::joinJoinKeyForEachTable need to handle theta-join. (jaehwa) (jhjung: rev 70567fcf47b5fdcc33a21be51c6de03e016ae4ff) tajo-core/src/test/java/org/apache/tajo/master/TestGlobalPlanner.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java CHANGES tajo-core/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              blrunner Jaehwa Jung
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development