Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1236

Remove slow 'new String' operation in parquet format

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.0
    • Component/s: Storage
    • Labels:
      None

      Description

      Currently, the parquet tuple converter creates ‘new String’ but TextDataum need the byte array. It is unnecessary operation.

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #489 (See https://builds.apache.org/job/Tajo-master-build/489/)
        TAJO-1236: Remove slow 'new String' operation in parquet format. (jinho) (jhkim: rev 661c7e216d664c71d2c889082703ecc6ae028dc3)

        • CHANGES
        • tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java
        • tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #489 (See https://builds.apache.org/job/Tajo-master-build/489/ ) TAJO-1236 : Remove slow 'new String' operation in parquet format. (jinho) (jhkim: rev 661c7e216d664c71d2c889082703ecc6ae028dc3) CHANGES tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #130 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/130/)
        TAJO-1236: Remove slow 'new String' operation in parquet format. (jinho) (jhkim: rev 661c7e216d664c71d2c889082703ecc6ae028dc3)

        • tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        • CHANGES
        • tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #130 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/130/ ) TAJO-1236 : Remove slow 'new String' operation in parquet format. (jinho) (jhkim: rev 661c7e216d664c71d2c889082703ecc6ae028dc3) tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java CHANGES tajo-storage/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java
        Hide
        jhkim Jinho Kim added a comment -

        Committed it
        Thank you for your quick review!

        Show
        jhkim Jinho Kim added a comment - Committed it Thank you for your quick review!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/292

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/292
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/292#issuecomment-66236385

        This improvement is straightforward.
        The patch looks good.
        +1!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/292#issuecomment-66236385 This improvement is straightforward. The patch looks good. +1!
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12685932/TAJO-1236.patch
        against master revision release-0.9.0-rc0-78-g8f68b4b.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings.

        -1 release audit. The applied patch generated 196 release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-storage.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/536//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/536//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/536//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/536//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685932/TAJO-1236.patch against master revision release-0.9.0-rc0-78-g8f68b4b. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 8 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 196 release audit warnings. +1 core tests. The patch passed unit tests in tajo-storage. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/536//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/536//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/536//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/536//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/292

        TAJO-1236: Remove slow 'new String' operation in parquet format

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-1236

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/292.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #292


        commit 8d6218ca98f6d80e5f63a5aa9839f0397d8a50ac
        Author: jhkim <jhkim@apache.org>
        Date: 2014-12-09T03:21:34Z

        TAJO-1236: Remove slow 'new String' operation in parquet format


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/292 TAJO-1236 : Remove slow 'new String' operation in parquet format You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1236 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/292.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #292 commit 8d6218ca98f6d80e5f63a5aa9839f0397d8a50ac Author: jhkim <jhkim@apache.org> Date: 2014-12-09T03:21:34Z TAJO-1236 : Remove slow 'new String' operation in parquet format

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development