Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-283 Add Table Partitioning
  3. TAJO-329

Implement physical operator to store in column-partitioned table.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Physical Operator
    • Labels:
      None

      Description

      We have to implement physical operators for partitioned table. First of all, I'm going to implement a physical operator to store in column-partitioned table. And I'll make another jira issue to implement a operator to load. After I fix all issue, I'm going to update HiveConverter and HCatalogStore.

      1. TAJO-329_2.patch
        21 kB
        Jaehwa Jung
      2. TAJO-329.patch
        18 kB
        Jaehwa Jung

        Activity

        Hide
        blrunner Jaehwa Jung added a comment -

        I uploaded the first patch. It just support to store on column partitioned table. To scan on column partitioned table will handle on TAJO-338 issue.

        Show
        blrunner Jaehwa Jung added a comment - I uploaded the first patch. It just support to store on column partitioned table. To scan on column partitioned table will handle on TAJO-338 issue.
        Hide
        hyunsik Hyunsik Choi added a comment -

        In overall, the patch looks good for me. I leave a couple of comments:

        • ScanNode contains some code about partitions. Even though it may be necessary later, this is not a part of this issue.
        • Could you please add unit tests for two or more partition columns to TestInsertQuery.java?
          • These unit tests are necessary.
        • ColumnPartitionedTableStoreExec::next() contains unused variable value.
        • It would be better to use datum.asChars() instead of datum.toString().
          • toString is not intended to output a pure value string.
        • I would like to recommend using StringBuilder rather than StringBuffer in those cases. StringBuffer is a thread-safe data structure, and it may be more costly than StringBuilder.
        Show
        hyunsik Hyunsik Choi added a comment - In overall, the patch looks good for me. I leave a couple of comments: ScanNode contains some code about partitions. Even though it may be necessary later, this is not a part of this issue. Could you please add unit tests for two or more partition columns to TestInsertQuery.java? These unit tests are necessary. ColumnPartitionedTableStoreExec::next() contains unused variable value . It would be better to use datum.asChars() instead of datum.toString(). toString is not intended to output a pure value string. I would like to recommend using StringBuilder rather than StringBuffer in those cases. StringBuffer is a thread-safe data structure, and it may be more costly than StringBuilder.
        Hide
        blrunner Jaehwa Jung added a comment -

        Thanks Hyunsik Choi. I agree entirely.
        I updated the patch. Please, check it again.

        Show
        blrunner Jaehwa Jung added a comment - Thanks Hyunsik Choi . I agree entirely. I updated the patch. Please, check it again.
        Hide
        hyunsik Hyunsik Choi added a comment -

        +1
        nice job. ship it!

        Show
        hyunsik Hyunsik Choi added a comment - +1 nice job. ship it!
        Hide
        blrunner Jaehwa Jung added a comment -

        Thanks Hyunsik Choi. I've just committed now.

        Show
        blrunner Jaehwa Jung added a comment - Thanks Hyunsik Choi . I've just committed now.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #607 (See https://builds.apache.org/job/Tajo-trunk-postcommit/607/)
        TAJO-329: Implement physical operator to store in column-partitioned table. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=22a6136d33da65f86319aead0a60c34140b11172)

        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
        • CHANGES.txt
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ColumnPartitionedTableStoreExec.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #607 (See https://builds.apache.org/job/Tajo-trunk-postcommit/607/ ) TAJO-329 : Implement physical operator to store in column-partitioned table. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=22a6136d33da65f86319aead0a60c34140b11172 ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java CHANGES.txt tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ColumnPartitionedTableStoreExec.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java

          People

          • Assignee:
            blrunner Jaehwa Jung
            Reporter:
            blrunner Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development