Hadoop Common
  1. Hadoop Common
  2. HADOOP-5213

BZip2CompressionOutputStream NullPointerException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.1
    • Fix Version/s: 0.19.2, 0.20.1, 0.21.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      BZip2CompressionOutputStream will throw a NullPointerException if the user creates a BZip2CompressionOutputStream and close it without writing out any data.

      1. HADOOP-5213.2.19.patch
        3 kB
        Zheng Shao
      2. HADOOP-5213.2.patch
        3 kB
        Zheng Shao
      3. HADOOP-5213.3.19.patch
        3 kB
        Zheng Shao
      4. HADOOP-5213.3.20.patch
        3 kB
        Zheng Shao
      5. HADOOP-5213.3.patch
        3 kB
        Zheng Shao

        Issue Links

          Activity

          Zheng Shao created issue -
          Zheng Shao made changes -
          Field Original Value New Value
          Link This issue blocks HADOOP-4918 [ HADOOP-4918 ]
          Hide
          Zheng Shao added a comment -

          This bug is introduced from HADOOP-4918

          Show
          Zheng Shao added a comment - This bug is introduced from HADOOP-4918
          Zheng Shao made changes -
          Attachment HADOOP-5213.1.patch [ 12399966 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.19.1 [ 12313473 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12399966/HADOOP-5213.1.patch
          against trunk revision 743296.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12399966/HADOOP-5213.1.patch against trunk revision 743296. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3833/console This message is automatically generated.
          Hide
          Nigel Daley added a comment -

          Please include a unit test.

          Show
          Nigel Daley added a comment - Please include a unit test.
          Hide
          Zheng Shao added a comment -

          Added unit test for empty inputs.

          Show
          Zheng Shao added a comment - Added unit test for empty inputs.
          Zheng Shao made changes -
          Attachment HADOOP-5213.2.patch [ 12400035 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12400035/HADOOP-5213.2.patch
          against trunk revision 743419.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12400035/HADOOP-5213.2.patch against trunk revision 743419. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/console This message is automatically generated.
          Show
          Zheng Shao added a comment - Checked the failed contrib test: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3835/testReport/org.apache.hadoop.chukwa.datacollection.agent/TestAgentConfig/testInitAdaptors_vs_Checkpoint/ It's completely unrelated.
          Hide
          Abdul Qadeer added a comment -

          I went through the change and it does not interfere with my bzip2 patch (https://issues.apache.org/jira/browse/HADOOP-4012)
          So a +1 as far as bzip2 stuff is concerned.

          Show
          Abdul Qadeer added a comment - I went through the change and it does not interfere with my bzip2 patch ( https://issues.apache.org/jira/browse/HADOOP-4012 ) So a +1 as far as bzip2 stuff is concerned.
          Zheng Shao made changes -
          Attachment HADOOP-5213.1.patch [ 12399966 ]
          Hide
          Zheng Shao added a comment -

          Patch for 0.19.

          Show
          Zheng Shao added a comment - Patch for 0.19.
          Zheng Shao made changes -
          Attachment HADOOP-5213.2.19.patch [ 12400209 ]
          Nigel Daley made changes -
          Fix Version/s 0.19.2 [ 12313650 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.19.1 [ 12313473 ]
          Hide
          Johan Oskarsson added a comment -

          To be really picky it seems the added "if (needsReset)" is redundant, since internalReset() already checks that boolean.

          Show
          Johan Oskarsson added a comment - To be really picky it seems the added "if (needsReset)" is redundant, since internalReset() already checks that boolean.
          Hide
          Zheng Shao added a comment -

          Agree. In the ideal case, the check of needsReset inside internalReset() should be "assert".
          But I guess this is OK considering all the overhead of remaking the patch and waiting for hudson.

          Show
          Zheng Shao added a comment - Agree. In the ideal case, the check of needsReset inside internalReset() should be "assert". But I guess this is OK considering all the overhead of remaking the patch and waiting for hudson.
          Hide
          dhruba borthakur added a comment -

          Hi folks, what is the status for this one? if the pach is ready, can one of you pl review it and submit it for HadoopQA tests? I would like this to get committed sooner rather than later.

          Show
          dhruba borthakur added a comment - Hi folks, what is the status for this one? if the pach is ready, can one of you pl review it and submit it for HadoopQA tests? I would like this to get committed sooner rather than later.
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.20.1 [ 12313866 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Abdul Qadeer made changes -
          Link This issue blocks HADOOP-4012 [ HADOOP-4012 ]
          Hide
          Zheng Shao added a comment -

          I provided the patch and it already went through Hadoop QA on 11/Feb/09 05:01 PM.
          Can we directly commit it? I have patch for both branch 0.19 and trunk.

          Show
          Zheng Shao added a comment - I provided the patch and it already went through Hadoop QA on 11/Feb/09 05:01 PM. Can we directly commit it? I have patch for both branch 0.19 and trunk.
          Hide
          Zheng Shao added a comment -

          For trunk. It's the same patch as HADOOP-5213.2.patch except some offsets are changed due to recent hadoop changes.

          Show
          Zheng Shao added a comment - For trunk. It's the same patch as HADOOP-5213 .2.patch except some offsets are changed due to recent hadoop changes.
          Zheng Shao made changes -
          Attachment HADOOP-5213.3.patch [ 12405727 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Zheng Shao added a comment -

          For branch 0.19.

          Show
          Zheng Shao added a comment - For branch 0.19.
          Zheng Shao made changes -
          Attachment HADOOP-5213.3.19.patch [ 12405728 ]
          Hide
          Zheng Shao added a comment -

          For branch 0.20, please use the same patch as trunk.

          Show
          Zheng Shao added a comment - For branch 0.20, please use the same patch as trunk.
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.19.1 [ 12313473 ]
          Affects Version/s 0.19.0 [ 12313211 ]
          Hide
          Zheng Shao added a comment -

          HADOOP-5213.3.20.patch is the same as HADOOP-5213.3.patch.
          Just add it to make it more clear.

          Show
          Zheng Shao added a comment - HADOOP-5213 .3.20.patch is the same as HADOOP-5213 .3.patch. Just add it to make it more clear.
          Zheng Shao made changes -
          Attachment HADOOP-5213.3.20.patch [ 12405730 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12405730/HADOOP-5213.3.20.patch
          against trunk revision 767122.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12405730/HADOOP-5213.3.20.patch against trunk revision 767122. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/217/console This message is automatically generated.
          Hide
          Abdul Qadeer added a comment -

          Now Hudson has given a +1, it would be nice to review and commit the patch by the Committers. My patch for HADOOP-4012 is waiting for this commit.

          Thanks.

          Show
          Abdul Qadeer added a comment - Now Hudson has given a +1, it would be nice to review and commit the patch by the Committers. My patch for HADOOP-4012 is waiting for this commit. Thanks.
          Hide
          dhruba borthakur added a comment -

          I am going to put this in the 0.20 branch. This is a major blocker for most sites using bzip2compression.

          Show
          dhruba borthakur added a comment - I am going to put this in the 0.20 branch. This is a major blocker for most sites using bzip2compression.
          dhruba borthakur made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Hide
          dhruba borthakur added a comment -

          I just committed this. Thanks Zheng!

          Show
          dhruba borthakur added a comment - I just committed this. Thanks Zheng!
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #824 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/824/ )
          Hide
          Hemanth Yamijala added a comment -

          Dhruba, the CHANGES.txt in branch 0.20 has the log message for this JIRA under 0.20.1. However, in the trunk, it is under 0.19.2. Shouldn't they be under the earliest version for all branches (in this case 0.19.2). I noticed these while trying to merge CHANGES.txt between trunk and 0.20. Can you please verify this ?

          Show
          Hemanth Yamijala added a comment - Dhruba, the CHANGES.txt in branch 0.20 has the log message for this JIRA under 0.20.1. However, in the trunk, it is under 0.19.2. Shouldn't they be under the earliest version for all branches (in this case 0.19.2). I noticed these while trying to merge CHANGES.txt between trunk and 0.20. Can you please verify this ?
          Zheng Shao made changes -
          Comment [ I am on vacation 5/2-5/10.

          For mobile ETL issues, please contact Ravi Grover.
          For HiPal issues, please contact Hao Liu.
          For Hive and any other issues, please contact Ashish Thusoo.

          ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks HADOOP-4012 [ HADOOP-4012 ]
          Gavin made changes -
          Link This issue is depended upon by HADOOP-4012 [ HADOOP-4012 ]

            People

            • Assignee:
              Zheng Shao
              Reporter:
              Zheng Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development