Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2961

Increase the default threadpool size for container launching in the application master.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      Currently the default threadpool size is 10 for launching containers in ContainerLauncherImpl. We should increase that to 100 for a reasonable default, so that container launching is not backed up by a small thread pool size.

      1. MAPREDUCE-2961.patch
        5 kB
        Arun C Murthy
      2. MAPREDUCE-2961-20110923.1.txt
        5 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-2961-20110923.txt
        3 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-2961.patch
        1 kB
        Mahadev konar
      5. MAPREDUCE-2961.patch
        1 kB
        Mahadev konar

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #25 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/25/)
        Merge -r 1175293:1175294 from trunk to branch-0.23 to fix MAPREDUCE-2961.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175295
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #25 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/25/ ) Merge -r 1175293:1175294 from trunk to branch-0.23 to fix MAPREDUCE-2961 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175295 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #20 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/20/)
        Merge -r 1175293:1175294 from trunk to branch-0.23 to fix MAPREDUCE-2961.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175295
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #20 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/20/ ) Merge -r 1175293:1175294 from trunk to branch-0.23 to fix MAPREDUCE-2961 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175295 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496379/MAPREDUCE-2961.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/851//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496379/MAPREDUCE-2961.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/851//console This message is automatically generated.
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks Vinod!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks Vinod!
        Arun C Murthy made changes -
        Attachment MAPREDUCE-2961.patch [ 12496379 ]
        Hide
        Arun C Murthy added a comment -

        Minor change - move default value to MRJobConfig.

        Show
        Arun C Murthy added a comment - Minor change - move default value to MRJobConfig.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496276/MAPREDUCE-2961-20110923.1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/839//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/839//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496276/MAPREDUCE-2961-20110923.1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/839//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/839//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Oh, BTW, I took Karam's help to see this patch's effect. Before this patch, we had 200 core-pool size for a 100,000 map-only jobs on ~350 nodes. With this patch, which automatically takes care of the thread-pool size(and sets to ~350 for this job in particular), we saw ~30% improvement on wall-clock time.

        Show
        Vinod Kumar Vavilapalli added a comment - Oh, BTW, I took Karam's help to see this patch's effect. Before this patch, we had 200 core-pool size for a 100,000 map-only jobs on ~350 nodes. With this patch, which automatically takes care of the thread-pool size(and sets to ~350 for this job in particular), we saw ~30% improvement on wall-clock time.
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        For very large clusters, one can up the limit(default 500) to increase scalability via an expert-level, non-documented configuration.

        The default stack-size for a thread is 500K, so given 2GB default vmem for AM, default upper limit of 500 should be fine.

        Show
        Vinod Kumar Vavilapalli added a comment - For very large clusters, one can up the limit(default 500) to increase scalability via an expert-level, non-documented configuration. The default stack-size for a thread is 500K, so given 2GB default vmem for AM, default upper limit of 500 should be fine.
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-2961-20110923.1.txt [ 12496276 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Updating patch, including the limit on the number of threads.

        With this patch, in most cases, the thread-pool should scale according to the need.

        Show
        Vinod Kumar Vavilapalli added a comment - Updating patch, including the limit on the number of threads. With this patch, in most cases, the thread-pool should scale according to the need.
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496232/MAPREDUCE-2961-20110923.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/835//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/835//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496232/MAPREDUCE-2961-20110923.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/835//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/835//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Anything above 100 threads in a overkill I think... plus each thread has a stack of 1MB. So, we shouldn't be using too many.

        Show
        Arun C Murthy added a comment - Anything above 100 threads in a overkill I think... plus each thread has a stack of 1MB. So, we shouldn't be using too many.
        Hide
        Mahadev konar added a comment -

        Vinod,
        Doesnt look like there is cap on the thread pool size. Should we put in cap at 300-400 or so?

        Show
        Mahadev konar added a comment - Vinod, Doesnt look like there is cap on the thread pool size. Should we put in cap at 300-400 or so?
        Hide
        Mahadev konar added a comment -

        nice patch, looks good to me! Will commit after hudson +1's.

        Show
        Mahadev konar added a comment - nice patch, looks good to me! Will commit after hudson +1's.
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-2961-20110923.txt [ 12496232 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Attaching patch that should fix this.

        The way it is, the pool size grows to the configured core-pool-size and stays there. We should set core-pool-size to be very low, set a max-pool-size and bound the queue size.

        We don't need to do this anymore, as the corepool itself is changed according to container-demand.

        Show
        Vinod Kumar Vavilapalli added a comment - Attaching patch that should fix this. The way it is, the pool size grows to the configured core-pool-size and stays there. We should set core-pool-size to be very low, set a max-pool-size and bound the queue size. We don't need to do this anymore, as the corepool itself is changed according to container-demand.
        Vinod Kumar Vavilapalli made changes -
        Assignee Mahadev konar [ mahadev ] Vinod Kumar Vavilapalli [ vinodkv ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        There are a couple of other bugs that need to be fixed related to ContainerLauncher's thread pool.

        • The way it is, the pool size grows to the configured core-pool-size and stays there. We should set core-pool-size to be very low, set a max-pool-size and bound the queue size.
        • We also pre-allocate threads which affects MRAppMaster initialization time.

        I am taking this up.

        Show
        Vinod Kumar Vavilapalli added a comment - There are a couple of other bugs that need to be fixed related to ContainerLauncher's thread pool. The way it is, the pool size grows to the configured core-pool-size and stays there. We should set core-pool-size to be very low, set a max-pool-size and bound the queue size. We also pre-allocate threads which affects MRAppMaster initialization time. I am taking this up.
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Forgot about this one.

        It is easy to set the number of threads dynamically. We just need to set it equal to the number of nodes in the system. We are looking at a scalability of 10,000 nodes, so I'd put a cap of 2000 threads, hardcoded.

        Show
        Vinod Kumar Vavilapalli added a comment - Forgot about this one. It is easy to set the number of threads dynamically. We just need to set it equal to the number of nodes in the system. We are looking at a scalability of 10,000 nodes, so I'd put a cap of 2000 threads, hardcoded.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496181/MAPREDUCE-2961.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/831//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/831//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496181/MAPREDUCE-2961.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/831//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/831//console This message is automatically generated.
        Mahadev konar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Mahadev konar added a comment -

        @Vinod,
        I do agree this is a hack but unless we see that it doesnt work, I probably think we should not spend too much time on estimating the right amount of threads. Thoughts?

        Show
        Mahadev konar added a comment - @Vinod, I do agree this is a hack but unless we see that it doesnt work, I probably think we should not spend too much time on estimating the right amount of threads. Thoughts?
        Mahadev konar made changes -
        Attachment MAPREDUCE-2961.patch [ 12496181 ]
        Hide
        Mahadev konar added a comment -

        the same patch on top of new changes.

        Show
        Mahadev konar added a comment - the same patch on top of new changes.
        Arun C Murthy made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Hide
        Arun C Murthy added a comment -

        Vinod, what heuristic do you have in mind?

        Show
        Arun C Murthy added a comment - Vinod, what heuristic do you have in mind?
        Hide
        Vinod Kumar Vavilapalli added a comment -

        I definitely think that instead of a config, this should be a function of #nodes and #tasks with defaults to 10. Will provide a patch for the same.

        Show
        Vinod Kumar Vavilapalli added a comment - I definitely think that instead of a config, this should be a function of #nodes and #tasks with defaults to 10. Will provide a patch for the same.
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Arun C Murthy added a comment -

        Mahadev, you need to rebase after MAPREDUCE-2864. Thanks.

        Show
        Arun C Murthy added a comment - Mahadev, you need to rebase after MAPREDUCE-2864 . Thanks.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12493691/MAPREDUCE-2961.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493691/MAPREDUCE-2961.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/665//console This message is automatically generated.
        Ted Yu made changes -
        Summary Increase the default threadpool fize for container launching in the application master. Increase the default threadpool size for container launching in the application master.
        Mahadev konar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Mahadev konar made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-2961.patch [ 12493691 ]
        Hide
        Mahadev konar added a comment -

        Trivial patch to increase the number of threads in the launcher so that we do not bottleneck on container launching.

        Show
        Mahadev konar added a comment - Trivial patch to increase the number of threads in the launcher so that we do not bottleneck on container launching.
        Mahadev konar created issue -

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Mahadev konar
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development