Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown.

      Description

      Daniel Dai found this. Here's what he says:

      I see hundreds of MRAppMaster process on my machine, and lots of tests fail for "Too many open files".

      1. MAPREDUCE-3586-20111220.txt
        2 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Here's the bit of stack trace, again from Daniel:

          "CompositeServiceShutdownHook for org.apache.hadoop.mapreduce.v2.app.MRAppMaster" prio=5 tid=1051e6000 nid=0x115ac5000 in Object.wait() [115ac4000]
             java.lang.Thread.State: WAITING (on object monitor)
              at java.lang.Object.wait(Native Method)
              - waiting on <7a5662ef0> (a java.lang.Thread)
              at java.lang.Thread.join(Thread.java:1210)
              - locked <7a5662ef0> (a java.lang.Thread)
              at java.lang.Thread.join(Thread.java:1263)
              at org.apache.hadoop.yarn.event.AsyncDispatcher.stop(AsyncDispatcher.java:104)
              at org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:95)
              - locked <7a54ada90> (a org.apache.hadoop.mapreduce.v2.app.MRAppMaster)
              at org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:85)
              - locked <7a54ada90> (a org.apache.hadoop.mapreduce.v2.app.MRAppMaster)
              at org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook.run(CompositeService.java:119)
          
          "AsyncDispatcher event handler" prio=5 tid=117084800 nid=0x115101000 in Object.wait() [115100000]
             java.lang.Thread.State: WAITING (on object monitor)
              at java.lang.Object.wait(Native Method)
              - waiting on <7a54cd298> (a org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook)
              at java.lang.Thread.join(Thread.java:1210)
              - locked <7a54cd298> (a org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook)
              at java.lang.Thread.join(Thread.java:1263)
              at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
              at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
              at java.lang.Shutdown.runHooks(Shutdown.java:79)
              at java.lang.Shutdown.sequence(Shutdown.java:123)
              at java.lang.Shutdown.exit(Shutdown.java:168)
              - locked <7faf9cd00> (a java.lang.Class for java.lang.Shutdown)
              at java.lang.Runtime.exit(Runtime.java:90)
              at java.lang.System.exit(System.java:921)
              at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.sysexit(MRAppMaster.java:366)
              at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler.handle(MRAppMaster.java:415)
              at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler.handle(MRAppMaster.java:369)
              at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:123)
              at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:82)
              at java.lang.Thread.run(Thread.java:680)
          
          Show
          Vinod Kumar Vavilapalli added a comment - Here's the bit of stack trace, again from Daniel: "CompositeServiceShutdownHook for org.apache.hadoop.mapreduce.v2.app.MRAppMaster" prio=5 tid=1051e6000 nid=0x115ac5000 in Object .wait() [115ac4000] java.lang. Thread .State: WAITING (on object monitor) at java.lang. Object .wait(Native Method) - waiting on <7a5662ef0> (a java.lang. Thread ) at java.lang. Thread .join( Thread .java:1210) - locked <7a5662ef0> (a java.lang. Thread ) at java.lang. Thread .join( Thread .java:1263) at org.apache.hadoop.yarn.event.AsyncDispatcher.stop(AsyncDispatcher.java:104) at org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:95) - locked <7a54ada90> (a org.apache.hadoop.mapreduce.v2.app.MRAppMaster) at org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:85) - locked <7a54ada90> (a org.apache.hadoop.mapreduce.v2.app.MRAppMaster) at org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook.run(CompositeService.java:119) "AsyncDispatcher event handler" prio=5 tid=117084800 nid=0x115101000 in Object .wait() [115100000] java.lang. Thread .State: WAITING (on object monitor) at java.lang. Object .wait(Native Method) - waiting on <7a54cd298> (a org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook) at java.lang. Thread .join( Thread .java:1210) - locked <7a54cd298> (a org.apache.hadoop.yarn.service.CompositeService$CompositeServiceShutdownHook) at java.lang. Thread .join( Thread .java:1263) at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79) at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24) at java.lang.Shutdown.runHooks(Shutdown.java:79) at java.lang.Shutdown.sequence(Shutdown.java:123) at java.lang.Shutdown.exit(Shutdown.java:168) - locked <7faf9cd00> (a java.lang. Class for java.lang.Shutdown) at java.lang. Runtime .exit( Runtime .java:90) at java.lang. System .exit( System .java:921) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.sysexit(MRAppMaster.java:366) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler.handle(MRAppMaster.java:415) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler.handle(MRAppMaster.java:369) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:123) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:82) at java.lang. Thread .run( Thread .java:680)
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Link This issue is related to PIG-2347 [ PIG-2347 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I've spent enough time, but still not sure why this is happening only in PIG testing and not on my local unit testing or the cluster testing we've been doing.

          Regardless, there is a stop-gap solution: Making the stop() of CompositeService reentrant so that components are only stopped once. Once we do that, we can prevent this issue of two threads waiting to join on each other completely.

          Show
          Vinod Kumar Vavilapalli added a comment - I've spent enough time, but still not sure why this is happening only in PIG testing and not on my local unit testing or the cluster testing we've been doing. Regardless, there is a stop-gap solution: Making the stop() of CompositeService reentrant so that components are only stopped once. Once we do that, we can prevent this issue of two threads waiting to join on each other completely.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Simple patch to avoid duplicate stop.

          Show
          Vinod Kumar Vavilapalli added a comment - Simple patch to avoid duplicate stop.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-3586-20111220.txt [ 12508131 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Arun C Murthy added a comment -

          +1

          Show
          Arun C Murthy added a comment - +1
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12508131/MAPREDUCE-3586-20111220.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1482//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1482//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12508131/MAPREDUCE-3586-20111220.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1482//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1482//console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Dan, can you please see if this works for you? Thanks!

          Show
          Vinod Kumar Vavilapalli added a comment - Dan, can you please see if this works for you? Thanks!
          Hide
          Daniel Dai added a comment -

          +1, didn't see this issue in Pig tests any more. Thanks Vinod!

          Show
          Daniel Dai added a comment - +1, didn't see this issue in Pig tests any more. Thanks Vinod!
          Hide
          Mahadev konar added a comment -

          +1 the patch looks good.

          Show
          Mahadev konar added a comment - +1 the patch looks good.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Thanks for the testing, Dan! I am going to push this in.

          Show
          Vinod Kumar Vavilapalli added a comment - Thanks for the testing, Dan! I am going to push this in.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Just committed this to trunk and merged it into branch 0.23.

          Show
          Vinod Kumar Vavilapalli added a comment - Just committed this to trunk and merged it into branch 0.23.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Release Note Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown.
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1538 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1538/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1538 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1538/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1466 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1466/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1466 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1466/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #308 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/308/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)
          svn merge -c 1221950 --ignore-ancestry ../../trunk/

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #308 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/308/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) svn merge -c 1221950 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #319 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/319/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)
          svn merge -c 1221950 --ignore-ancestry ../../trunk/

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #319 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/319/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) svn merge -c 1221950 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1487 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1487/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1487 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1487/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #330 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/330/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)
          svn merge -c 1221950 --ignore-ancestry ../../trunk/

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #330 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/330/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) svn merge -c 1221950 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #115 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/115/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)
          svn merge -c 1221950 --ignore-ancestry ../../trunk/

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #115 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/115/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) svn merge -c 1221950 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #902 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/902/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #902 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/902/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/136/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)
          svn merge -c 1221950 --ignore-ancestry ../../trunk/

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/136/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) svn merge -c 1221950 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221951 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #935 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/935/)
          MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv)

          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #935 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/935/ ) MAPREDUCE-3586 . Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1221950 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/CompositeService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestCompositeService.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #320 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/320/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #320 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/320/ )
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development