Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-283 Add Table Partitioning
  3. TAJO-329

Implement physical operator to store in column-partitioned table.

        Details

        • Type: Sub-task
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.8.0
        • Fix Version/s: 0.8.0
        • Component/s: Physical Operator
        • Labels:
          None

          Description

          We have to implement physical operators for partitioned table. First of all, I'm going to implement a physical operator to store in column-partitioned table. And I'll make another jira issue to implement a operator to load. After I fix all issue, I'm going to update HiveConverter and HCatalogStore.

          1. Text File
            TAJO-329_2.patch
            21 kB
            Jaehwa Jung
          2. Text File
            TAJO-329.patch
            18 kB
            Jaehwa Jung

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            blrunner Jaehwa Jung added a comment -

            I uploaded the first patch. It just support to store on column partitioned table. To scan on column partitioned table will handle on TAJO-338 issue.

            Show
            blrunner Jaehwa Jung added a comment - I uploaded the first patch. It just support to store on column partitioned table. To scan on column partitioned table will handle on TAJO-338 issue.
            Hide
            Permalink
            hyunsik Hyunsik Choi added a comment -

            In overall, the patch looks good for me. I leave a couple of comments:

            • ScanNode contains some code about partitions. Even though it may be necessary later, this is not a part of this issue.
            • Could you please add unit tests for two or more partition columns to TestInsertQuery.java?
              • These unit tests are necessary.
            • ColumnPartitionedTableStoreExec::next() contains unused variable value.
            • It would be better to use datum.asChars() instead of datum.toString().
              • toString is not intended to output a pure value string.
            • I would like to recommend using StringBuilder rather than StringBuffer in those cases. StringBuffer is a thread-safe data structure, and it may be more costly than StringBuilder.
            Show
            hyunsik Hyunsik Choi added a comment - In overall, the patch looks good for me. I leave a couple of comments: ScanNode contains some code about partitions. Even though it may be necessary later, this is not a part of this issue. Could you please add unit tests for two or more partition columns to TestInsertQuery.java? These unit tests are necessary. ColumnPartitionedTableStoreExec::next() contains unused variable value . It would be better to use datum.asChars() instead of datum.toString(). toString is not intended to output a pure value string. I would like to recommend using StringBuilder rather than StringBuffer in those cases. StringBuffer is a thread-safe data structure, and it may be more costly than StringBuilder.
            Hide
            Permalink
            blrunner Jaehwa Jung added a comment -

            Thanks Hyunsik Choi. I agree entirely.
            I updated the patch. Please, check it again.

            Show
            blrunner Jaehwa Jung added a comment - Thanks Hyunsik Choi . I agree entirely. I updated the patch. Please, check it again.
            Hide
            Permalink
            hyunsik Hyunsik Choi added a comment -

            +1
            nice job. ship it!

            Show
            hyunsik Hyunsik Choi added a comment - +1 nice job. ship it!
            Hide
            Permalink
            blrunner Jaehwa Jung added a comment -

            Thanks Hyunsik Choi. I've just committed now.

            Show
            blrunner Jaehwa Jung added a comment - Thanks Hyunsik Choi . I've just committed now.
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Tajo-trunk-postcommit #607 (See https://builds.apache.org/job/Tajo-trunk-postcommit/607/)
            TAJO-329: Implement physical operator to store in column-partitioned table. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=22a6136d33da65f86319aead0a60c34140b11172)

            • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
            • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
            • CHANGES.txt
            • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ColumnPartitionedTableStoreExec.java
            • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #607 (See https://builds.apache.org/job/Tajo-trunk-postcommit/607/ ) TAJO-329 : Implement physical operator to store in column-partitioned table. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=22a6136d33da65f86319aead0a60c34140b11172 ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java CHANGES.txt tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ColumnPartitionedTableStoreExec.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java

              People

              • Assignee:
                blrunner Jaehwa Jung
                Reporter:
                blrunner Jaehwa Jung
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development