Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2, performance
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Hide
      Removes two sets of previously available config properties:

      1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
      2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )

      In favor of two new, generically named properties:

      1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
      2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.

      Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.

      Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
      Show
      Removes two sets of previously available config properties: 1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and, 2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb ) In favor of two new, generically named properties: 1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers. 2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers. Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides. Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
    • Target Version/s:

      Description

      After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB.

      This is just a configuration change.

      [P.s.]:

      • Currently min/max alloc is set at a per-scheduler config level, which makes no sense as there's no way to run multiple schedulers anyway. Switch configs to use a generic RM-config.
      • The min/max alloc configs aren't documented and we ought to document it (i.e. MAPREDUCE-4027)
      • 1 GB is perhaps too high for a slot's minimum. While job defaults can be left at such values, we should lower the minimum alloc to 128 MB to allow special requests of low memory out of the box itself. Shouldn't impact MR App in any way.
      1. MAPREDUCE-3812-20120205.txt
        10 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-3812-20120206.txt
        10 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-3812-20120206.1.txt
        19 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-3812.patch
        21 kB
        Arun C Murthy
      5. MAPREDUCE-3812.patch
        25 kB
        Arun C Murthy
      6. MAPREDUCE-3812.patch
        21 kB
        Harsh J
      7. MAPREDUCE-3812.patch
        20 kB
        Harsh J
      8. MAPREDUCE-3812.patch
        21 kB
        Harsh J

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Component/s mrv2 [ 12314301 ]
          Component/s performance [ 12316500 ]
          Vinod Kumar Vavilapalli made changes -
          Assignee Vinod Kumar Vavilapalli [ vinodkv ]
          Fix Version/s 0.23.1 [ 12318883 ]
          Affects Version/s 0.23.0 [ 12315570 ]
          Description After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB.

          This is just a configuration change.
          Vinod Kumar Vavilapalli made changes -
          Link This issue is related to MAPREDUCE-3719 [ MAPREDUCE-3719 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This helps gridmix performance also due to the extra slots that are freed up.

          Show
          Vinod Kumar Vavilapalli added a comment - This helps gridmix performance also due to the extra slots that are freed up.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Digging more, I realized we've been testing with 1.5 slot size for normal tasks and the same 1.5GB for AM since a long time. I am going to change the defaults to reflect what we've been testing with.

          Show
          Vinod Kumar Vavilapalli added a comment - Digging more, I realized we've been testing with 1.5 slot size for normal tasks and the same 1.5GB for AM since a long time. I am going to change the defaults to reflect what we've been testing with.
          Vinod Kumar Vavilapalli made changes -
          Summary Change default AM slot size to be 1GB Change default memory slot sizes to be 1.5GB
          Hide
          Vinod Kumar Vavilapalli added a comment -
          • Changed default minimum-allocation-mb to be 1536, maximum-allocation-mb to be 6*1536, default total memory per node to be 6*1536.
          • Changed default slot size for AM to be 1536, heap size to be 1024mb.
          • Also removed the per-scheduler min and max allocation mb configurations. These are common concepts across schedulers.
          Show
          Vinod Kumar Vavilapalli added a comment - Changed default minimum-allocation-mb to be 1536, maximum-allocation-mb to be 6*1536, default total memory per node to be 6*1536. Changed default slot size for AM to be 1536, heap size to be 1024mb. Also removed the per-scheduler min and max allocation mb configurations. These are common concepts across schedulers.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-3812-20120205.txt [ 12513379 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1781//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1781//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1782//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1782//console This message is automatically generated.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Correct patch applicable to trunk.

          Show
          Vinod Kumar Vavilapalli added a comment - Correct patch applicable to trunk.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-3812-20120206.txt [ 12513476 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513476/MAPREDUCE-3812-20120206.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
          org.apache.hadoop.yarn.server.resourcemanager.TestFifoScheduler
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513476/MAPREDUCE-3812-20120206.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup org.apache.hadoop.yarn.server.resourcemanager.TestFifoScheduler org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//console This message is automatically generated.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Fixing tests.

          Show
          Vinod Kumar Vavilapalli added a comment - Fixing tests.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-3812-20120206.1.txt [ 12513504 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513504/MAPREDUCE-3812-20120206.1.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed the unit tests build

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513504/MAPREDUCE-3812-20120206.1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//console This message is automatically generated.
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Arun C Murthy added a comment -

          Fixed unit tests.

          Show
          Arun C Murthy added a comment - Fixed unit tests.
          Arun C Murthy made changes -
          Attachment MAPREDUCE-3812.patch [ 12513549 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513549/MAPREDUCE-3812.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513549/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//console This message is automatically generated.
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Arun C Murthy added a comment -

          Fixed the test.

          Show
          Arun C Murthy added a comment - Fixed the test.
          Arun C Murthy made changes -
          Attachment MAPREDUCE-3812.patch [ 12513566 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12513566/MAPREDUCE-3812.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.mapred.TestMiniMRBringup
          org.apache.hadoop.mapred.TestMiniMRChildTask
          org.apache.hadoop.mapred.TestReduceFetch
          org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
          org.apache.hadoop.mapred.TestJobCounters
          org.apache.hadoop.mapreduce.TestChild
          org.apache.hadoop.mapred.TestMiniMRClientCluster
          org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
          org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
          org.apache.hadoop.mapred.TestClientRedirect
          org.apache.hadoop.mapred.TestLazyOutput
          org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
          org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
          org.apache.hadoop.mapreduce.v2.TestRMNMInfo
          org.apache.hadoop.mapred.TestJobSysDirWithDFS
          org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513566/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestMiniMRBringup org.apache.hadoop.mapred.TestMiniMRChildTask org.apache.hadoop.mapred.TestReduceFetch org.apache.hadoop.mapred.TestReduceFetchFromPartialMem org.apache.hadoop.mapred.TestJobCounters org.apache.hadoop.mapreduce.TestChild org.apache.hadoop.mapred.TestMiniMRClientCluster org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter org.apache.hadoop.mapred.TestClientRedirect org.apache.hadoop.mapred.TestLazyOutput org.apache.hadoop.mapreduce.TestMapReduceLazyOutput org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath org.apache.hadoop.mapreduce.v2.TestRMNMInfo org.apache.hadoop.mapred.TestJobSysDirWithDFS org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This patch needs more work. Moving to 0.23.2.

          Show
          Vinod Kumar Vavilapalli added a comment - This patch needs more work. Moving to 0.23.2.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Fix Version/s 0.23.2 [ 12319851 ]
          Fix Version/s 0.23.1 [ 12318883 ]
          Hide
          Harsh J added a comment -

          Setting a target version instead of a fix version.

          Show
          Harsh J added a comment - Setting a target version instead of a fix version.
          Harsh J made changes -
          Fix Version/s 0.23.2 [ 12319851 ]
          Target Version/s 0.23.3 [ 12320060 ]
          Harsh J made changes -
          Link This issue is superceded by MAPREDUCE-4026 [ MAPREDUCE-4026 ]
          Hide
          Harsh J added a comment -

          Didn't realize this patch changed the way allocs would get read, or that it was doing more than just changing default AM memory requirements. Would save me a lot of work I've otherwise been doing on MAPREDUCE-4026 and MAPREDUCE-4027.

          I'll take over this one since it looks to have gone stale. Thanks for revamping the whole default way of reading the allocs - much simpler way now. No idea why there was per-scheduler configs before when you can't use more than one scheduler anyway.

          Show
          Harsh J added a comment - Didn't realize this patch changed the way allocs would get read, or that it was doing more than just changing default AM memory requirements. Would save me a lot of work I've otherwise been doing on MAPREDUCE-4026 and MAPREDUCE-4027 . I'll take over this one since it looks to have gone stale. Thanks for revamping the whole default way of reading the allocs - much simpler way now. No idea why there was per-scheduler configs before when you can't use more than one scheduler anyway.
          Harsh J made changes -
          Assignee Vinod Kumar Vavilapalli [ vinodkv ] Harsh J [ qwertymaniac ]
          Harsh J made changes -
          Target Version/s 0.23.3 [ 12320060 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Hide
          Harsh J added a comment -

          Updated title and description

          Show
          Harsh J added a comment - Updated title and description
          Harsh J made changes -
          Summary Change default memory slot sizes to be 1.5GB Lower default allocation sizes, fix allocation configurations and document them
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Description After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB.

          This is just a configuration change.
          After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB.

          This is just a configuration change.

          [P.s.]:
          - Currently min/max alloc is set at a per-scheduler config level, which makes no sense as there's no way to run multiple schedulers anyway. Switch configs to use a generic RM-config.
          - The min/max alloc configs aren't documented and we ought to document it (i.e. MAPREDUCE-4027)
          - 1 GB is perhaps too high for a slot's minimum. While job defaults can be left at such values, we should lower the minimum alloc to 128 MB to allow special requests of low memory out of the box itself. Shouldn't impact MR App in any way.
          Hide
          Harsh J added a comment -

          Here is a fresh rewrite patch as the last one available here was out of sync and I figured its faster rewriting changes and gathering MAPREDUCE-4026 and MAPREDUCE-4027 rather than trying to resolve the out-of-sync conflicts.

          I've not touched some tests as I felt they were unnecessary. I ran 4 tests for this change: TestFifoScheduler, TestRMWebServices, TestRM, TestApplicationCleanup and all of them pass.

          Show
          Harsh J added a comment - Here is a fresh rewrite patch as the last one available here was out of sync and I figured its faster rewriting changes and gathering MAPREDUCE-4026 and MAPREDUCE-4027 rather than trying to resolve the out-of-sync conflicts. I've not touched some tests as I felt they were unnecessary. I ran 4 tests for this change: TestFifoScheduler, TestRMWebServices, TestRM, TestApplicationCleanup and all of them pass.
          Harsh J made changes -
          Attachment MAPREDUCE-3812.patch [ 12523646 ]
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Hide
          Harsh J added a comment -

          Ah and yes this is an incompatible change as it changes the way min/max alloc configs are read. With this patch, no more per-scheduler min/max alloc configs exist.

          Show
          Harsh J added a comment - Ah and yes this is an incompatible change as it changes the way min/max alloc configs are read. With this patch, no more per-scheduler min/max alloc configs exist.
          Harsh J made changes -
          Hadoop Flags Incompatible change [ 10342 ]
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Harsh J made changes -
          Link This issue is duplicated by MAPREDUCE-4026 [ MAPREDUCE-4026 ]
          Harsh J made changes -
          Link This issue is duplicated by MAPREDUCE-4027 [ MAPREDUCE-4027 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12523646/MAPREDUCE-3812.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.TestContainerManagerSecurity
          org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens
          org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
          org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
          org.apache.hadoop.mapred.TestClientRedirect
          org.apache.hadoop.mapreduce.security.TestJHSSecurity

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2268//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2268//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523646/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.TestContainerManagerSecurity org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator org.apache.hadoop.mapred.TestClientRedirect org.apache.hadoop.mapreduce.security.TestJHSSecurity +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2268//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2268//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Off the failures, only TestRMContainerAllocator looked relevant. Fixed that test by removing an unnecessary init override (could've also called super.init but the whole method override was unneeded, I had it carried from the earlier patch).

          Here are new test results, from those that failed, per project:

          YARN Server:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.472 sec
          Running org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.826 sec
          Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.453 sec
          Running org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.21 sec
          Running org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.997 sec
          
          Results :
          
          Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
          

          MapReduce Client App:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
          Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.106 sec
          
          Results :
          
          Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
          

          MR JobClient:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.mapred.TestClientRedirect
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.422 sec
          Running org.apache.hadoop.mapreduce.security.TestJHSSecurity
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.899 sec
          
          Results :
          
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
          

          All failed tests have passed locally. On jenkins it looks like some other test failing to shutdown YARN services in teardown methods is causing this issue. Resubmitting.

          Show
          Harsh J added a comment - Off the failures, only TestRMContainerAllocator looked relevant. Fixed that test by removing an unnecessary init override (could've also called super.init but the whole method override was unneeded, I had it carried from the earlier patch). Here are new test results, from those that failed, per project: YARN Server : ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.472 sec Running org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.826 sec Running org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.453 sec Running org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.21 sec Running org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.997 sec Results : Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 MapReduce Client App : ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.106 sec Results : Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 MR JobClient : ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.mapred.TestClientRedirect Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.422 sec Running org.apache.hadoop.mapreduce.security.TestJHSSecurity Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.899 sec Results : Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 All failed tests have passed locally. On jenkins it looks like some other test failing to shutdown YARN services in teardown methods is causing this issue. Resubmitting.
          Harsh J made changes -
          Attachment MAPREDUCE-3812.patch [ 12523664 ]
          Hide
          Harsh J added a comment -

          One micro change: Updating mapred-default's AM memory value as a new default entry was added via MAPREDUCE-3947 recently.

          Show
          Harsh J added a comment - One micro change: Updating mapred-default's AM memory value as a new default entry was added via MAPREDUCE-3947 recently.
          Harsh J made changes -
          Attachment MAPREDUCE-3812.patch [ 12523665 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12523665/MAPREDUCE-3812.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.TestContainerManagerSecurity
          org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens
          org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
          org.apache.hadoop.mapred.TestClientRedirect
          org.apache.hadoop.mapreduce.security.TestJHSSecurity

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2269//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2269//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523665/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.TestContainerManagerSecurity org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs org.apache.hadoop.mapred.TestClientRedirect org.apache.hadoop.mapreduce.security.TestJHSSecurity +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2269//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2269//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12523665/MAPREDUCE-3812.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.TestContainerManagerSecurity

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2270//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2270//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523665/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.TestContainerManagerSecurity +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2270//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2270//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Failing test is now unrelated to patch. Please review!

          Show
          Harsh J added a comment - Failing test is now unrelated to patch. Please review!
          Hide
          Robert Joseph Evans added a comment -

          The changes look OK to me. +1. I am not really sure about the need to make the different configurations common, but looking at the patch for the fair scheduler it uses the same concept so I am OK with the change.

          Show
          Robert Joseph Evans added a comment - The changes look OK to me. +1. I am not really sure about the need to make the different configurations common, but looking at the patch for the fair scheduler it uses the same concept so I am OK with the change.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2190 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2190/)
          MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2190 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2190/ ) MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2116 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2116/)
          MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2116 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2116/ ) MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2132 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2132/)
          MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2132 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2132/ ) MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Hide
          Robert Joseph Evans added a comment -

          Thanks Harsh,

          I put this into branch-0.23, branch-2, and trunk

          Show
          Robert Joseph Evans added a comment - Thanks Harsh, I put this into branch-0.23, branch-2, and trunk
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Fix Version/s 0.23.3 [ 12320060 ]
          Fix Version/s 2.0.0 [ 12320354 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #237 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/237/)
          svn merge -c 1329282 FIXES: MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329285)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329285
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #237 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/237/ ) svn merge -c 1329282 FIXES: MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329285) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329285 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1024 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1024/)
          MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1024 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1024/ ) MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1059 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1059/)
          MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1059 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1059/ ) MAPREDUCE-3812 . Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) (Revision 1329282) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329282 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
          Harsh J made changes -
          Release Note Removed two sets of previously available config properties:

          1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
          2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )

          In favor of two new, generically named properties:

          1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
          2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.

          Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.

          Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Harsh J made changes -
          Release Note Removed two sets of previously available config properties:

          1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
          2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )

          In favor of two new, generically named properties:

          1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
          2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.

          Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.

          Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
          Removes two sets of previously available config properties:

          1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
          2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )

          In favor of two new, generically named properties:

          1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
          2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.

          Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.

          Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
          Target Version/s trunk, 2.0.0 [ 12320360, 12320354 ] 2.0.0, trunk [ 12320354, 12320360 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322471 ]
          Fix Version/s 2.0.0-alpha [ 12320354 ]
          Fix Version/s 3.0.0 [ 12320355 ]

            People

            • Assignee:
              Harsh J
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development