Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1249

mapreduce.reduce.shuffle.read.timeout's default value should be 3 minutes, in mapred-default.xml

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: task
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      mapreduce.reduce.shuffle.read.timeout has a value of 30,000 (30 seconds) in mapred-default.xml, whereas the default value in Fetcher code is 3 minutes.
      It should be 3 minutes by default, as it was in pre MAPREDUCE-353.

      1. patch-1249-1.txt
        1 kB
        Amareshwari Sriramadasu
      2. patch-1249.txt
        0.5 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Amareshwari Sriramadasu added a comment -

          Patch changing the value in mapred-default.xml

          Show
          Amareshwari Sriramadasu added a comment - Patch changing the value in mapred-default.xml
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12426393/patch-1249.txt
          against trunk revision 884832.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12426393/patch-1249.txt against trunk revision 884832. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/279/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          It is documented that the parameters mapreduce.reduce.shuffle.connect.timeout and mapreduce.reduce.shuffle.read.timeout are cluster-wide parameters. But, actually they are job-level parameters. I think the tag "Expert" conveys the fact that users are not supposed to play with it.

          Show
          Amareshwari Sriramadasu added a comment - It is documented that the parameters mapreduce.reduce.shuffle.connect.timeout and mapreduce.reduce.shuffle.read.timeout are cluster-wide parameters. But, actually they are job-level parameters. I think the tag "Expert" conveys the fact that users are not supposed to play with it.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch updating the doc.

          Show
          Amareshwari Sriramadasu added a comment - Patch updating the doc.
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failures are unrelated to the patch. Resubmitting for hudson

          Show
          Amareshwari Sriramadasu added a comment - Test failures are unrelated to the patch. Resubmitting for hudson
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12426497/patch-1249-1.txt
          against trunk revision 885530.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12426497/patch-1249-1.txt against trunk revision 885530. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/155/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          -1 contrib tests

          Is due to MAPREDUCE-1124 and MAPREDUCE-1245

          -1 tests included.

          The change is in mapred-default.xml, Ran a job manually and verified the value is correctly picked up.

          Show
          Amareshwari Sriramadasu added a comment - -1 contrib tests Is due to MAPREDUCE-1124 and MAPREDUCE-1245 -1 tests included. The change is in mapred-default.xml, Ran a job manually and verified the value is correctly picked up.
          Hide
          Chris Douglas added a comment -

          +1

          I committed this. Thanks, Amareshwari!

          Show
          Chris Douglas added a comment - +1 I committed this. Thanks, Amareshwari!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #142 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/142/)
          . Update config default value for socket read timeout to
          match code default. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #142 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/142/ ) . Update config default value for socket read timeout to match code default. Contributed by Amareshwari Sriramadasu
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #164 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/164/)
          . Update config default value for socket read timeout to
          match code default. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #164 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/164/ ) . Update config default value for socket read timeout to match code default. Contributed by Amareshwari Sriramadasu

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development