Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1621

Streaming's TextOutputReader.getLastOutput throws NPE if it has never read any output

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/streaming
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If TextOutputReader.readKeyValue() has never successfully read a line, then its bytes member will be left null. Thus when logging a task failure, PipeMapRed.getContext() can trigger an NPE when it calls outReader_.getLastOutput().

      1. patch-1621.txt
        2 kB
        Amareshwari Sriramadasu

        Activity

        Hide
        Amareshwari Sriramadasu added a comment -

        Here is a patch fixing the NPE.

        Currently, NPE can be see from TestStreamingBadRecords logs. After the fix, I don't see the NPE.

        Show
        Amareshwari Sriramadasu added a comment - Here is a patch fixing the NPE. Currently, NPE can be see from TestStreamingBadRecords logs. After the fix, I don't see the NPE.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12449214/patch-1621.txt
        against trunk revision 962682.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449214/patch-1621.txt against trunk revision 962682. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/597/console This message is automatically generated.
        Hide
        Amareshwari Sriramadasu added a comment -

        Many tests failed because of NoClassDefFoundError. Re-submitting to hudson

        Show
        Amareshwari Sriramadasu added a comment - Many tests failed because of NoClassDefFoundError. Re-submitting to hudson
        Hide
        Amareshwari Sriramadasu added a comment -

        Seems hudson is not picking up submitted patches, I ran test-patch and ant test on my machine. All tests passed except the known failures in trunk (MAPREDUCE-1834 and MAPREDUCE-1925).

        Show
        Amareshwari Sriramadasu added a comment - Seems hudson is not picking up submitted patches, I ran test-patch and ant test on my machine. All tests passed except the known failures in trunk ( MAPREDUCE-1834 and MAPREDUCE-1925 ).
        Hide
        Ravi Gummadi added a comment -

        Patch looks good.
        +1

        Show
        Ravi Gummadi added a comment - Patch looks good. +1
        Hide
        Amareshwari Sriramadasu added a comment -

        I just committed this.

        Show
        Amareshwari Sriramadasu added a comment - I just committed this.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development