Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11418

Property "io.compression.codec.lzo.class" does not work with other value besides default

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: io
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      From following code, seems "io.compression.codec.lzo.class" does not work for other codec besides default. Hadoop will always treat it as defaultClazz. I think it is a bug. Please let me know if this is a work as design thing. Thanks
      77 private static final String defaultClazz =
      78 "org.apache.hadoop.io.compress.LzoCodec";

      82 public synchronized boolean isSupported() {
      83 if (!checked) {
      84 checked = true;
      85 String extClazz =
      86 (conf.get(CONF_LZO_CLASS) == null ? System
      87 .getProperty(CONF_LZO_CLASS) : null);
      88 String clazz = (extClazz != null) ? extClazz : defaultClazz;

      1. HADOOP-11418.005.patch
        3 kB
        Akira Ajisaka
      2. HADOOP-11418.006.patch
        3 kB
        Akira Ajisaka
      3. HADOOP-11418.patch
        0.9 kB
        fang fang chen
      4. HADOOP-11418-004.patch
        3 kB
        Akira Ajisaka
      5. HADOOP-11418-1.patch
        1 kB
        fang fang chen
      6. HADOOP-11418-2.patch
        1 kB
        fang fang chen
      7. HADOOP-11418-3.patch
        1 kB
        fang fang chen

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12687712/HADOOP-11418.patch
          against trunk revision e996a1b.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.crypto.random.TestOsSecureRandom

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12687712/HADOOP-11418.patch against trunk revision e996a1b. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.crypto.random.TestOsSecureRandom Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5288//console This message is automatically generated.
          Hide
          fang fang chen fang fang chen added a comment -

          Could someone give a glance to this issue and comment?

          Show
          fang fang chen fang fang chen added a comment - Could someone give a glance to this issue and comment?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12687712/HADOOP-11418.patch
          against trunk revision d1c6acc.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5826//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5826//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12687712/HADOOP-11418.patch against trunk revision d1c6acc. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5826//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5826//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks fang fang chen for the report and the patch. Nice catch. The fix looks good to me. Two comments:
          1. The code lookups conf twice. I'm thinking it's better to do in a single lookup as follows:

                    String extClazz = conf.get(CONF_LZO_CLASS);
                    if (extClazz == null) {
                      extClazz = System.getProperty(CONF_LZO_CLASS);
                    }
          

          2. Would you add a regression test for this? I'm thinking the test will

          • Set "io.compression.codec.lzo.class" to a codec other than the default lzo codec (ex. o.a.h.io.compress.DefaultCodec)
          • Call Algorithm.LZO.isSupported()
          • Call Algorithm.LZO.getCodec() and verify the specified codec is obtained.
          Show
          ajisakaa Akira Ajisaka added a comment - Thanks fang fang chen for the report and the patch. Nice catch. The fix looks good to me. Two comments: 1. The code lookups conf twice. I'm thinking it's better to do in a single lookup as follows: String extClazz = conf.get(CONF_LZO_CLASS); if (extClazz == null ) { extClazz = System .getProperty(CONF_LZO_CLASS); } 2. Would you add a regression test for this? I'm thinking the test will Set "io.compression.codec.lzo.class" to a codec other than the default lzo codec (ex. o.a.h.io.compress.DefaultCodec) Call Algorithm.LZO.isSupported() Call Algorithm.LZO.getCodec() and verify the specified codec is obtained.
          Hide
          fang fang chen fang fang chen added a comment -

          Sure, thanks for looking at this JIRA, Akira. I generated new patch based on current trunk. And the regression test has been made with pig before, here is the process and log:
          1. Add following settings in hadoop core-site.xml:
          <property>
          <name>io.compression.codecs</name>
          <value>com.ibm.biginsights.compress.CmxCodec</value>
          </property>
          <property>
          <name>io.compression.codec.lzo.class</name>
          <value>com.ibm.biginsights.compress.CmxCodec</value>
          </property>
          2. Launch pig grunt with properties:
          -Dpig.tmpfilecompression=true -Dpig.tmpfilecompression.codec=cmx -Dpig.additional.jars=ibm-compression.jar:libcmxbiginsights.so

          Together with patch in PIG-4341, test case passed with following massage:
          2014-12-22 17:37:24,453 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 2
          2014-12-22 17:37:24,550 [main] INFO org.apache.hadoop.io.file.tfile.Compression - Trying to load Lzo codec class: com.ibm.biginsights.compress.CmxCodec
          2014-12-22 17:37:24,554 [main] INFO com.ibm.biginsights.compress.NativeCodeLoader - Loaded the native library
          2014-12-22 17:37:24,554 [main] INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor [.cmx]

          Show
          fang fang chen fang fang chen added a comment - Sure, thanks for looking at this JIRA, Akira. I generated new patch based on current trunk. And the regression test has been made with pig before, here is the process and log: 1. Add following settings in hadoop core-site.xml: <property> <name>io.compression.codecs</name> <value>com.ibm.biginsights.compress.CmxCodec</value> </property> <property> <name>io.compression.codec.lzo.class</name> <value>com.ibm.biginsights.compress.CmxCodec</value> </property> 2. Launch pig grunt with properties: -Dpig.tmpfilecompression=true -Dpig.tmpfilecompression.codec=cmx -Dpig.additional.jars=ibm-compression.jar:libcmxbiginsights.so Together with patch in PIG-4341 , test case passed with following massage: 2014-12-22 17:37:24,453 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 2 2014-12-22 17:37:24,550 [main] INFO org.apache.hadoop.io.file.tfile.Compression - Trying to load Lzo codec class: com.ibm.biginsights.compress.CmxCodec 2014-12-22 17:37:24,554 [main] INFO com.ibm.biginsights.compress.NativeCodeLoader - Loaded the native library 2014-12-22 17:37:24,554 [main] INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor [.cmx]
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12702130/HADOOP-11418-1.patch
          against trunk revision 4228de9.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5829//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5829//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702130/HADOOP-11418-1.patch against trunk revision 4228de9. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5829//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5829//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          And the regression test has been made with pig before, here is the process and log:

          The log looks good to me, however, I'd like to add a regression test in hadoop side also. I'm not sure PIG-4341 will be merged soon.

          +              (extClazz_conf == null ? System.getProperty(CONF_LZO_CLASS) : extClazz_conf);
          
          • The line contains unnecessary parentheses. Would you remove them?
          • The line is longer than 80 characters. Would you render it within 80 characters?
          Show
          ajisakaa Akira Ajisaka added a comment - And the regression test has been made with pig before, here is the process and log: The log looks good to me, however, I'd like to add a regression test in hadoop side also. I'm not sure PIG-4341 will be merged soon. + (extClazz_conf == null ? System .getProperty(CONF_LZO_CLASS) : extClazz_conf); The line contains unnecessary parentheses. Would you remove them? The line is longer than 80 characters. Would you render it within 80 characters?
          Hide
          fang fang chen fang fang chen added a comment -

          About "unnecessary parentheses", do you mean it should me like this:
          + String extClazz = extClazz_conf == null ? System
          + .getProperty(CONF_LZO_CLASS) : extClazz_conf;

          It looks like a little strange.

          Let me provide 2 patch files, HADOOP-11418-3.patch remove parentheses and HADOOP-11418-2.patch keep the parentheses.

          Show
          fang fang chen fang fang chen added a comment - About "unnecessary parentheses", do you mean it should me like this: + String extClazz = extClazz_conf == null ? System + .getProperty(CONF_LZO_CLASS) : extClazz_conf; It looks like a little strange. Let me provide 2 patch files, HADOOP-11418 -3.patch remove parentheses and HADOOP-11418 -2.patch keep the parentheses.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12702381/HADOOP-11418-3.patch
          against trunk revision 29bb689.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5839//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5839//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702381/HADOOP-11418-3.patch against trunk revision 29bb689. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5839//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5839//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          It looks like a little strange.

          I agree with you. Rethinking this, the below looks clear to me. What do you think?

            String extClazz = (extClazz_conf == null) ?
                System.getProperty(CONF_LZO_CLASS) : extClazz_conf;
          

          Would you please add a regression test? If you don't want to add it, I'd like to succeed your work.

          Show
          ajisakaa Akira Ajisaka added a comment - It looks like a little strange. I agree with you. Rethinking this, the below looks clear to me. What do you think? String extClazz = (extClazz_conf == null ) ? System .getProperty(CONF_LZO_CLASS) : extClazz_conf; Would you please add a regression test? If you don't want to add it, I'd like to succeed your work.
          Hide
          fang fang chen fang fang chen added a comment -

          Please go ahead to add a regression test. Thanks

          Show
          fang fang chen fang fang chen added a comment - Please go ahead to add a regression test. Thanks
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks fang fang chen. Added a regression test.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks fang fang chen . Added a regression test.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12707171/HADOOP-11418-004.patch
          against trunk revision b351086.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5995//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5995//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707171/HADOOP-11418-004.patch against trunk revision b351086. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5995//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5995//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          v5 patch:

          • Fix inconsistency between
                      String extClazz = (extClazz_conf == null) ?
                          System.getProperty(CONF_LZO_CLASS) : extClazz_conf;
            

            and

                      String clazz = (extClazz != null) ? extClazz : defaultClazz;
            

            . One is using ==, the other is using !=.

          Show
          ajisakaa Akira Ajisaka added a comment - v5 patch: Fix inconsistency between String extClazz = (extClazz_conf == null ) ? System .getProperty(CONF_LZO_CLASS) : extClazz_conf; and String clazz = (extClazz != null ) ? extClazz : defaultClazz; . One is using == , the other is using != .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 17s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 10m 3s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 10s The applied patch generated 1 new checkstyle issues (total was 4, now 5).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 23m 34s Tests passed in hadoop-common.
              62m 6s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12734427/HADOOP-11418.005.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0e4f108
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 17s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 10s The applied patch generated 1 new checkstyle issues (total was 4, now 5). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 23m 34s Tests passed in hadoop-common.     62m 6s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734427/HADOOP-11418.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0e4f108 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6792/console This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Fixed checkstyle issue.

          Show
          ajisakaa Akira Ajisaka added a comment - Fixed checkstyle issue.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 44s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 4s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 41s Tests passed in hadoop-common.
              59m 53s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12734565/HADOOP-11418.006.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2b6bcfd
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 4s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 41s Tests passed in hadoop-common.     59m 53s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734565/HADOOP-11418.006.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2b6bcfd hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6795/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 43s trunk passed
          +1 compile 6m 17s trunk passed with JDK v1.8.0_74
          +1 compile 6m 47s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 33s trunk passed
          +1 javadoc 0m 52s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 5m 35s the patch passed with JDK v1.8.0_74
          +1 javac 5m 35s the patch passed
          +1 compile 6m 39s the patch passed with JDK v1.7.0_95
          +1 javac 6m 39s the patch passed
          +1 checkstyle 0m 20s the patch passed
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 50s the patch passed
          +1 javadoc 0m 51s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          -1 unit 6m 23s hadoop-common in the patch failed with JDK v1.8.0_74.
          -1 unit 6m 41s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 23s Patch generated 2 ASF License warnings.
          57m 36s



          Reason Tests
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12734565/HADOOP-11418.006.patch
          JIRA Issue HADOOP-11418
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c40fa2056d76 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a107cee
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 6m 17s trunk passed with JDK v1.8.0_74 +1 compile 6m 47s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 33s trunk passed +1 javadoc 0m 52s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 5m 35s the patch passed with JDK v1.8.0_74 +1 javac 5m 35s the patch passed +1 compile 6m 39s the patch passed with JDK v1.7.0_95 +1 javac 6m 39s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 51s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 -1 unit 6m 23s hadoop-common in the patch failed with JDK v1.8.0_74. -1 unit 6m 41s hadoop-common in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 23s Patch generated 2 ASF License warnings. 57m 36s Reason Tests JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12734565/HADOOP-11418.006.patch JIRA Issue HADOOP-11418 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c40fa2056d76 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a107cee Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8904/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Akira Ajisaka Thanks for your work. This issue has been assigned to me. I've reviewed the patch and it looks good. Could you please tell me what the next step is so that we can fix and close this issue?

          Show
          yuanbo Yuanbo Liu added a comment - Akira Ajisaka Thanks for your work. This issue has been assigned to me. I've reviewed the patch and it looks good. Could you please tell me what the next step is so that we can fix and close this issue?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for your comment. The latest patch needs another committer's +1 because I wrote it. Hi Masatake Iwasaki, would you please review the latest patch?

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for your comment. The latest patch needs another committer's +1 because I wrote it. Hi Masatake Iwasaki , would you please review the latest patch?
          Hide
          iwasakims Masatake Iwasaki added a comment -

          +1 from me. Committing this.

          Show
          iwasakims Masatake Iwasaki added a comment - +1 from me. Committing this.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9663 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9663/)
          HADOOP-11418. Property "io.compression.codec.lzo.class" does not work (iwasakims: rev bec5b4cd8c1fb27a5b707d601b508224a9c107ed)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestCompression.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9663 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9663/ ) HADOOP-11418 . Property "io.compression.codec.lzo.class" does not work (iwasakims: rev bec5b4cd8c1fb27a5b707d601b508224a9c107ed) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestCompression.java
          Hide
          iwasakims Masatake Iwasaki added a comment -

          Committed. Thanks, Yuanbo Liu, fang fang chen and Akira Ajisaka.

          Show
          iwasakims Masatake Iwasaki added a comment - Committed. Thanks, Yuanbo Liu , fang fang chen and Akira Ajisaka .
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Masatake!

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Masatake!
          Hide
          yuanbo Yuanbo Liu added a comment -

          Thanks for your great work, Akira and Masatake

          Show
          yuanbo Yuanbo Liu added a comment - Thanks for your great work, Akira and Masatake

            People

            • Assignee:
              yuanbo Yuanbo Liu
              Reporter:
              fang fang chen fang fang chen
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development