Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-879

Some data is missing in the case of BROADCAST JOIN and multi-column partition.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Labels:
      None

      Description

      If the data directory is the following, some data is missing.

      /tajo/warehouse/table1/year=2014/month=01/hour=12/part-01-00000
      /tajo/warehouse/table1/year=2014/month=02/hour=12/part-01-00000
      

      SeqScanExec uses a last partition column path's name as a broadcast table's cache key.
      In this case the table is partitioned by year, month, hour. So the cache key is "hour=12" which is not unique.
      It should be fixed.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user babokim opened a pull request:

        https://github.com/apache/tajo/pull/43

        TAJO-879: Some data is missing in the case of BROADCAST JOIN and multi-column partition.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/babokim/tajo TAJO-879

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/43.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #43


        commit 81d811a8549ac8e63c43a72a179a78efe5e694f2
        Author: 김형준 <babokim@babokim-mbp.server.gruter.com>
        Date: 2014-06-19T02:32:59Z

        TAJO-879: Some data is missing in the case of BROADCAST JOIN and multi-column partition.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user babokim opened a pull request: https://github.com/apache/tajo/pull/43 TAJO-879 : Some data is missing in the case of BROADCAST JOIN and multi-column partition. You can merge this pull request into a Git repository by running: $ git pull https://github.com/babokim/tajo TAJO-879 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/43.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #43 commit 81d811a8549ac8e63c43a72a179a78efe5e694f2 Author: 김형준 <babokim@babokim-mbp.server.gruter.com> Date: 2014-06-19T02:32:59Z TAJO-879 : Some data is missing in the case of BROADCAST JOIN and multi-column partition.
        Hide
        blrunner Jaehwa Jung added a comment -

        +1

        Hi kim hyoung jun

        Thank you for your contribution.
        You patch looks good, and I just updated TestJoinBroadcast::testBroadcastMultiColumnPartitionTable as follows:

        • remove unnecessary comments
        • Adding result file data

        I'll commit it to master branch now.

        Cheers
        Jaehwa

        Show
        blrunner Jaehwa Jung added a comment - +1 Hi kim hyoung jun Thank you for your contribution. You patch looks good, and I just updated TestJoinBroadcast::testBroadcastMultiColumnPartitionTable as follows: remove unnecessary comments Adding result file data I'll commit it to master branch now. Cheers Jaehwa
        Hide
        blrunner Jaehwa Jung added a comment -

        I've just committed it to the master branch.

        Show
        blrunner Jaehwa Jung added a comment - I've just committed it to the master branch.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #252 (See https://builds.apache.org/job/Tajo-master-build/252/)
        TAJO-879: Some data is missing in the case of BROADCAST JOIN and multi-column partition. (Hyoungjun Kim via jaehwa) (jhjung: rev 8883f9fc28a51aa9db6242206f75db49043e176b)

        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java
        • tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
        • tajo-core/src/test/resources/results/TestJoinBroadcast/testBroadcastMultiColumnPartitionTable.result
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #252 (See https://builds.apache.org/job/Tajo-master-build/252/ ) TAJO-879 : Some data is missing in the case of BROADCAST JOIN and multi-column partition. (Hyoungjun Kim via jaehwa) (jhjung: rev 8883f9fc28a51aa9db6242206f75db49043e176b) tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java CHANGES tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-core/src/test/resources/results/TestJoinBroadcast/testBroadcastMultiColumnPartitionTable.result
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/43#issuecomment-46943834

        This issue was committed, but it was not closed automatically. Could you close this ticket?

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/43#issuecomment-46943834 This issue was committed, but it was not closed automatically. Could you close this ticket?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user babokim closed the pull request at:

        https://github.com/apache/tajo/pull/43

        Show
        githubbot ASF GitHub Bot added a comment - Github user babokim closed the pull request at: https://github.com/apache/tajo/pull/43

          People

          • Assignee:
            hjkim Hyoungjun Kim
            Reporter:
            hjkim Hyoungjun Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development