Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1126

Join condition including functions throws IllegalArgumentException.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      MergeJoin with function eval throws the following IllegalArgumentException.

      select
        n1.n_nationkey,
        n1.n_name,
        n2.n_name
      from nation n1 join (select * from nation union select * from nation) n2 on substr(n1.n_name, 1, 4) = substr(n2.n_name, 1, 4)
      order by n1.n_nationkey;
      
      java.lang.IllegalArgumentException: At least one sort key must be specified
      	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
      	at org.apache.tajo.engine.planner.logical.SortNode.setSortSpecs(SortNode.java:35)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createMergeInnerJoin(PhysicalPlannerImpl.java:443)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createBestInnerJoinPlan(PhysicalPlannerImpl.java:433)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createInnerJoinPlan(PhysicalPlannerImpl.java:377)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createJoinPlan(PhysicalPlannerImpl.java:292)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createPlanRecursive(PhysicalPlannerImpl.java:218)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createPlanRecursive(PhysicalPlannerImpl.java:207)
      	at org.apache.tajo.engine.planner.PhysicalPlannerImpl.createPlan(PhysicalPlannerImpl.java:90)
      	at org.apache.tajo.worker.TajoQueryEngine.createPlan(TajoQueryEngine.java:40)
      	at org.apache.tajo.worker.Task.run(Task.java:442)
      	at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
      	at java.lang.Thread.run(Thread.java:744)
      

      The above query is from TestJoinQuery.testComplexJoinCondition7() test case. Currently TestJoinQuery runs with a parameterized option, but the parameter is not transmitted to UnitRequestImpl. So all test case run with HashJoin parameter. For this reason the above error is not detected. We should also fix the test case bug.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user hyunsik opened a pull request:

        https://github.com/apache/tajo/pull/209

        TAJO-1126: Join condition including functions throws IllegalArgumentExce...

        ...ption.

        Thank you for enclosing the reproducible query and detailed bug reports.

        In addition to above description, I discussed this problem with Hyoungjun in offline chat. He informed me the bug where the parameterized unit tests in TestJoin does not work. The fact that QueryUnitRequesttImpl newly creates TajoConf instance instead of taking existing TajoConf causes this bug. So, given parameters were lost during unit tests. This problem is hiding the unit test failures. As he mentioned, I could find about 20 test failures in TestJoin when I fixed this parameters loss problem.

        I found the bug from PlannerUtil::isJoinQual. I improved this method to take two schemas of both relations for join, and I also enabled the method to ensure by checking schemas if both expressions composes a join condition. As a result, all the failures in TestJoin are passed.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/hyunsik/tajo TAJO-1126

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/209.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #209


        commit 0e19341b4061a441a1e2a349cde78a24bfa281cd
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2014-10-24T05:48:24Z

        TAJO-1126: Join condition including functions throws IllegalArgumentException.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/209 TAJO-1126 : Join condition including functions throws IllegalArgumentExce... ...ption. Thank you for enclosing the reproducible query and detailed bug reports. In addition to above description, I discussed this problem with Hyoungjun in offline chat. He informed me the bug where the parameterized unit tests in TestJoin does not work. The fact that QueryUnitRequesttImpl newly creates TajoConf instance instead of taking existing TajoConf causes this bug. So, given parameters were lost during unit tests. This problem is hiding the unit test failures. As he mentioned, I could find about 20 test failures in TestJoin when I fixed this parameters loss problem. I found the bug from PlannerUtil::isJoinQual. I improved this method to take two schemas of both relations for join, and I also enabled the method to ensure by checking schemas if both expressions composes a join condition. As a result, all the failures in TestJoin are passed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1126 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/209.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #209 commit 0e19341b4061a441a1e2a349cde78a24bfa281cd Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-10-24T05:48:24Z TAJO-1126 : Join condition including functions throws IllegalArgumentException.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/209#issuecomment-60492886

        rebased.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/209#issuecomment-60492886 rebased.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/209#issuecomment-60551982

        +1 Looks great to me.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/209#issuecomment-60551982 +1 Looks great to me.
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed the latest patch to master branch.

        Show
        hyunsik Hyunsik Choi added a comment - committed the latest patch to master branch.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/209

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/209
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-CODEGEN-build #62 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/62/)
        TAJO-1126: Join condition including functions throws IllegalArgumentException. (hyunsik: rev d9ba02bc1adebf12660d0af192d3d3c5441e7cd1)

        • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalTreeUtil.java
        • tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequest.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
        • tajo-common/src/main/java/org/apache/tajo/OverridableConf.java
        • CHANGES
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java
        • tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-CODEGEN-build #62 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/62/ ) TAJO-1126 : Join condition including functions throws IllegalArgumentException. (hyunsik: rev d9ba02bc1adebf12660d0af192d3d3c5441e7cd1) tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalTreeUtil.java tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequest.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-common/src/main/java/org/apache/tajo/OverridableConf.java CHANGES tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #420 (See https://builds.apache.org/job/Tajo-master-build/420/)
        TAJO-1126: Join condition including functions throws IllegalArgumentException. (hyunsik: rev d9ba02bc1adebf12660d0af192d3d3c5441e7cd1)

        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalTreeUtil.java
        • tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequest.java
        • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
        • tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
        • tajo-common/src/main/java/org/apache/tajo/OverridableConf.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java
        • CHANGES
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #420 (See https://builds.apache.org/job/Tajo-master-build/420/ ) TAJO-1126 : Join condition including functions throws IllegalArgumentException. (hyunsik: rev d9ba02bc1adebf12660d0af192d3d3c5441e7cd1) tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalTreeUtil.java tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequest.java tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-core/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-common/src/main/java/org/apache/tajo/OverridableConf.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java CHANGES

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hjkim Hyoungjun Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development