Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Storage
    • Labels:
      None

      Description

      • If the block metadata(TAJO-84) is disabled in hadoop, the splits is calculated wrong. It caused by wrong TajoConf.
      • We don't need the max split size. It should be dfs.blocksize.

        Issue Links

          Activity

          Hide
          jhkim Jinho Kim added a comment - - edited

          Created a review request against branch master in reviewboard
          https://reviews.apache.org/r/17699

          Show
          jhkim Jinho Kim added a comment - - edited Created a review request against branch master in reviewboard https://reviews.apache.org/r/17699
          Hide
          jhkim Jinho Kim added a comment -

          Yongjun Park
          Could you please test on amazon S3 ?

          Show
          jhkim Jinho Kim added a comment - Yongjun Park Could you please test on amazon S3 ?
          Hide
          tajoqa Tajo QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12626847/TAJO-582.patch
          against master revision 51919cf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-common tajo-storage.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/103//testReport/
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/103//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12626847/TAJO-582.patch against master revision 51919cf. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-common tajo-storage. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/103//testReport/ Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/103//console This message is automatically generated.
          Hide
          yongjun Yongjun Park added a comment -

          [~jinossy] I'm testing on EC2 with S3. It seems to work well.
          I'll share a result of this at TAJO-577

          Show
          yongjun Yongjun Park added a comment - [~jinossy] I'm testing on EC2 with S3. It seems to work well. I'll share a result of this at TAJO-577
          Hide
          jhkim Jinho Kim added a comment -

          Thanks, Yongjun!

          Show
          jhkim Jinho Kim added a comment - Thanks, Yongjun!
          Hide
          blrunner Jaehwa Jung added a comment -

          +1.
          It looks good overall, and 'mvn clean install -Phcatalog-0.12.0' finished successfully.
          Ship it.

          Show
          blrunner Jaehwa Jung added a comment - +1. It looks good overall, and 'mvn clean install -Phcatalog-0.12.0' finished successfully. Ship it.
          Hide
          jhkim Jinho Kim added a comment -

          Thank you for the review. I've just committed it.

          Show
          jhkim Jinho Kim added a comment - Thank you for the review. I've just committed it.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #56 (See https://builds.apache.org/job/Tajo-master-build/56/)
          TAJO-582: Invalid split calculation. (jinho) (jinossy: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=2761436b2725ee983d28ad44c7f83475abd09628)

          • tajo-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java
          • tajo-storage/src/test/java/org/apache/tajo/storage/TestFileSystems.java
          • CHANGES.txt
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-storage/src/test/resources/storage-default.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #56 (See https://builds.apache.org/job/Tajo-master-build/56/ ) TAJO-582 : Invalid split calculation. (jinho) (jinossy: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=2761436b2725ee983d28ad44c7f83475abd09628 ) tajo-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java tajo-storage/src/test/java/org/apache/tajo/storage/TestFileSystems.java CHANGES.txt tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-storage/src/test/resources/storage-default.xml

            People

            • Assignee:
              jhkim Jinho Kim
              Reporter:
              jhkim Jinho Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development