Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-2077

Join condition causes incorrect result, when a table has an empty row file

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0
    • Fix Version/s: 0.12.0, 0.11.2
    • Component/s: Physical Operator
    • Labels:
      None

      Description

      MergeScanner termination condition is null tuple. if file fragment is still exist and first fragment returns null tuple, other fragments is ignored.

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-0.11.2-build #173 (See https://builds.apache.org/job/Tajo-0.11.2-build/173/)
        TAJO-2077: Join condition causes incorrect result, when a table has an (jhkim: rev a8f62b918f93997b32110400b350e64ad0459caf)

        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1
        • CHANGES
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java
        • tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java
        • tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-0.11.2-build #173 (See https://builds.apache.org/job/Tajo-0.11.2-build/173/ ) TAJO-2077 : Join condition causes incorrect result, when a table has an (jhkim: rev a8f62b918f93997b32110400b350e64ad0459caf) tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1 CHANGES tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1 tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3 tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2 tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2 tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3 tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #1090 (See https://builds.apache.org/job/Tajo-master-build/1090/)
        TAJO-2077: Join condition causes incorrect result, when a table has an (jhkim: rev 81ab265fdb77a2eedc0c9b4dccc2826b461daf56)

        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2
        • tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
        • CHANGES
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java
        • tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #1090 (See https://builds.apache.org/job/Tajo-master-build/1090/ ) TAJO-2077 : Join condition causes incorrect result, when a table has an (jhkim: rev 81ab265fdb77a2eedc0c9b4dccc2826b461daf56) tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3 tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2 tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3 tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java CHANGES tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2 tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1 tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1 tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql
        Hide
        jhkim Jinho Kim added a comment -

        committed it.
        Thanks for your quick review!

        Show
        jhkim Jinho Kim added a comment - committed it. Thanks for your quick review!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #683 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/683/)
        TAJO-2077: Join condition causes incorrect result, when a table has an (jhkim: rev 81ab265fdb77a2eedc0c9b4dccc2826b461daf56)

        • CHANGES
        • tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1
        • tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation
        • tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2
        • tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #683 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/683/ ) TAJO-2077 : Join condition causes incorrect result, when a table has an (jhkim: rev 81ab265fdb77a2eedc0c9b4dccc2826b461daf56) CHANGES tajo-core-tests/src/test/resources/results/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.result tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer3 tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_customer_parquet_ddl.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestInnerJoinQuery.java tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation3 tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation_24rows tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation2 tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/MergeJoinExec.java tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation1 tajo-core-tests/src/test/resources/queries/TestJoinQuery/create_nation_parquet_ddl.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer_5rows tajo-core-tests/src/test/resources/dataset/TestJoinQuery/nation_parquet/nation tajo-core-tests/src/test/resources/queries/TestInnerJoinQuery/testBroadcastJoinWithEmptyRows.sql tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer2 tajo-core-tests/src/test/resources/dataset/TestJoinQuery/customer_parquet/customer1
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/963

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/963
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/963#issuecomment-185025304

        +1 LGTM!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/963#issuecomment-185025304 +1 LGTM!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/963#issuecomment-185006369

        Test failure is not related this PR.
        refer to https://issues.apache.org/jira/browse/TAJO-2078

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/963#issuecomment-185006369 Test failure is not related this PR. refer to https://issues.apache.org/jira/browse/TAJO-2078
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/963

        TAJO-2077: Join condition causes incorrect result, when a table has an empty row file.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-2077

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/963.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #963


        commit 58b4cc249e5621b87edbf3f192f78d4443e63418
        Author: Jinho Kim <jhkim@apache.org>
        Date: 2016-02-17T02:51:13Z

        TAJO-2077: Join condition causes incorrect result, when a table has an empty row file.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/963 TAJO-2077 : Join condition causes incorrect result, when a table has an empty row file. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-2077 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/963.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #963 commit 58b4cc249e5621b87edbf3f192f78d4443e63418 Author: Jinho Kim <jhkim@apache.org> Date: 2016-02-17T02:51:13Z TAJO-2077 : Join condition causes incorrect result, when a table has an empty row file.

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development