Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      Exception should be logged properly when there is any while deserializing Shuffle Header information .

            if (mapId == null || mapOutput == null) {
              LOG.info("fetcher#" + id + " failed to read map header" + 
                       mapId + " decomp: " + 
                       decompressedLength + ", " + compressedLength, ioe);
              return false;
            }
          
      1. MAPREDUCE-3423-1.patch
        1 kB
        Bhallamudi Venkata Siva Kamesh
      2. MAPREDUCE-3423.patch
        2 kB
        Bhallamudi Venkata Siva Kamesh

        Activity

        Hide
        Bhallamudi Venkata Siva Kamesh added a comment -

        Hi Arun,
        Thanks for looking into this. I have not removed null check for mapOutput. I have removed only null check for mapId and mapId won't be null by the time outer catch block catches any ioe (considering this patch).

        Show
        Bhallamudi Venkata Siva Kamesh added a comment - Hi Arun, Thanks for looking into this. I have not removed null check for mapOutput. I have removed only null check for mapId and mapId won't be null by the time outer catch block catches any ioe (considering this patch).
        Hide
        Arun C Murthy added a comment -

        Kamesh - you've removed the null check for mapOutput?

        Show
        Arun C Murthy added a comment - Kamesh - you've removed the null check for mapOutput?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505303/MAPREDUCE-3423-1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505303/MAPREDUCE-3423-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//console This message is automatically generated.
        Hide
        Bhallamudi Venkata Siva Kamesh added a comment -

        Also, in the first part, wouldnt it be simpler if it was something like

        updated this.

        Show
        Bhallamudi Venkata Siva Kamesh added a comment - Also, in the first part, wouldnt it be simpler if it was something like updated this.
        Hide
        Bhallamudi Venkata Siva Kamesh added a comment -

        Am not sure I get the changes in the second part of the patch:

        I updated this part, thinking mapid won't be null, handling this in the first part and returning false in the error scenarios.This exception happens when MergeManager tries to allocate location for the map output,so, logging the same.

        Show
        Bhallamudi Venkata Siva Kamesh added a comment - Am not sure I get the changes in the second part of the patch: I updated this part, thinking mapid won't be null, handling this in the first part and returning false in the error scenarios.This exception happens when MergeManager tries to allocate location for the map output,so, logging the same.
        Hide
        Mahadev konar added a comment -

        Bhallamudi,
        Am not sure I get the changes in the second part of the patch:

        - if (mapId == null || mapOutput == null) {
        + if (mapOutput == null) {
        

        Also, the in the first part, wouldnt it be simpler if it was something like:

        try{
        } catch(IllegalArgument){
        } catch (IOException) {
        } catch(Exception) {
        } 
        
        Show
        Mahadev konar added a comment - Bhallamudi, Am not sure I get the changes in the second part of the patch: - if (mapId == null || mapOutput == null ) { + if (mapOutput == null ) { Also, the in the first part, wouldnt it be simpler if it was something like: try { } catch (IllegalArgument){ } catch (IOException) { } catch (Exception) { }
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12504087/MAPREDUCE-3423.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504087/MAPREDUCE-3423.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//console This message is automatically generated.
        Hide
        Bhallamudi Venkata Siva Kamesh added a comment -

        Attaching a patch with proper logging

        Show
        Bhallamudi Venkata Siva Kamesh added a comment - Attaching a patch with proper logging

          People

          • Assignee:
            Bhallamudi Venkata Siva Kamesh
            Reporter:
            Bhallamudi Venkata Siva Kamesh
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development