Hadoop YARN
  1. Hadoop YARN
  2. YARN-128 RM Restart
  3. YARN-231

Add FS-based persistent store implementation for RMStateStore

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      Add store that write RM state data to FileSystem

      1. YARN-231.1.patch
        40 kB
        Bikas Saha
      2. YARN-231.2.patch
        51 kB
        Bikas Saha
      3. YARN-231.3.FS.patch
        29 kB
        Bikas Saha

        Issue Links

          Activity

          Hide
          Bikas Saha added a comment -

          This patch applies on top of the patches for YARN-229 and YARN-230

          Show
          Bikas Saha added a comment - This patch applies on top of the patches for YARN-229 and YARN-230
          Hide
          Bikas Saha added a comment -

          FileSystem based implementation allows any FileSystem to act as a store destination. Specifically HDFS can be used and has been tested.

          Show
          Bikas Saha added a comment - FileSystem based implementation allows any FileSystem to act as a store destination. Specifically HDFS can be used and has been tested.
          Hide
          Bikas Saha added a comment -

          Both store have a similar design. Each app data and attempt data is stored in its own znode/file. The names of the znodes/files are the app and attempt id's. This creates discrete units of writes for each store operation and reduces potential areas of contention for downstream HA work.
          For loading the stored state, each app and attempt is parsed into its object. Attempts are then placed into collections for their corresponding apps and orphans are discarded.

          Show
          Bikas Saha added a comment - Both store have a similar design. Each app data and attempt data is stored in its own znode/file. The names of the znodes/files are the app and attempt id's. This creates discrete units of writes for each store operation and reduces potential areas of contention for downstream HA work. For loading the stored state, each app and attempt is parsed into its object. Attempts are then placed into collections for their corresponding apps and orphans are discarded.
          Hide
          Bikas Saha added a comment -

          Tests verify that the config has been correctly read and load/store operations happen as expected. The flow should be easy to follow.

          Show
          Bikas Saha added a comment - Tests verify that the config has been correctly read and load/store operations happen as expected. The flow should be easy to follow.
          Hide
          Tom White added a comment -

          Some initial feedback on FileSystemRMStateStore (I haven't looked at the ZK store):

          • There are references to znodes in FileSystemRMStateStore.
          • Failure to read a file in FileSystemRMStateStore should not cause the whole recovery process to fail, just that particular application or attempt.
          • The literals "application_" and "appattempt_" should be made into constants that live in ApplicationId and ApplicationAttemptId.
          Show
          Tom White added a comment - Some initial feedback on FileSystemRMStateStore (I haven't looked at the ZK store): There are references to znodes in FileSystemRMStateStore. Failure to read a file in FileSystemRMStateStore should not cause the whole recovery process to fail, just that particular application or attempt. The literals "application_" and "appattempt_" should be made into constants that live in ApplicationId and ApplicationAttemptId.
          Hide
          Bikas Saha added a comment -

          At this point I have tried to make the logic fail safe keeping in mind imminent HA work where load/store operations need to make sure that partial operations dont happen. Errors could either be because of store error or loss of master status. Loss of master status may actually manifest as a store error in many cases. Hence I have chosen to bail out on every error in the store. Of course, this may change to allow partial reads if such optimizations are deemed safe. Does that help clarify?
          Thats also partly responsible for why I am currently throwing Exception everywhere. I am yet not sure about all different exceptions that might be validly thrown by different store implementations. In the near term, I do expect that these might narrow down at least to IOException for a pure store error (e.g. NN is unavailable) and HAException that would clearly show that loss of master status has occurred. Until some preliminary HA work provides more inputs to this end, I thought I would leave the Exceptions generic for now. All interfaces are marked Evolving for this reason.
          My next patch should address the remaining feedback and any other new ones that you might have.

          Show
          Bikas Saha added a comment - At this point I have tried to make the logic fail safe keeping in mind imminent HA work where load/store operations need to make sure that partial operations dont happen. Errors could either be because of store error or loss of master status. Loss of master status may actually manifest as a store error in many cases. Hence I have chosen to bail out on every error in the store. Of course, this may change to allow partial reads if such optimizations are deemed safe. Does that help clarify? Thats also partly responsible for why I am currently throwing Exception everywhere. I am yet not sure about all different exceptions that might be validly thrown by different store implementations. In the near term, I do expect that these might narrow down at least to IOException for a pure store error (e.g. NN is unavailable) and HAException that would clearly show that loss of master status has occurred. Until some preliminary HA work provides more inputs to this end, I thought I would leave the Exceptions generic for now. All interfaces are marked Evolving for this reason. My next patch should address the remaining feedback and any other new ones that you might have.
          Hide
          Tom White added a comment -

          Errors could either be because of store error or loss of master status.

          Do you mean loss of master status of the RM doing the store? I wouldn't expect the store to know about the master's status, since that would be handled at a higher level (i.e. by the RM itself). So IOException would be sufficient, at least for this JIRA.

          All interfaces are marked Evolving for this reason.

          It would be better to use Unstable until the RM HA work is done. Evolving would mean that you couldn't change the exceptions between 2.0.3 and 2.0.4 (say).

          Show
          Tom White added a comment - Errors could either be because of store error or loss of master status. Do you mean loss of master status of the RM doing the store? I wouldn't expect the store to know about the master's status, since that would be handled at a higher level (i.e. by the RM itself). So IOException would be sufficient, at least for this JIRA. All interfaces are marked Evolving for this reason. It would be better to use Unstable until the RM HA work is done. Evolving would mean that you couldn't change the exceptions between 2.0.3 and 2.0.4 (say).
          Hide
          Bikas Saha added a comment -

          No. I mean RMStateStoreImpl catching store exceptions and figuring out whether master status has been lost or its an unrelated store error. e.g. An error saying the store has disallowed access vs and error saying the store is unavailable. The first would be an HA exception meaning the RM should stop vs the second exception would mean a loss of store which may be tolerable when storing non-critical information like node health. So, I would like to narrow down from Exception into a better subset as we progress instead of making everything look like an IOException now.

          Good point. I will mark stuff as Unstable instead for both this and YARN-230

          Show
          Bikas Saha added a comment - No. I mean RMStateStoreImpl catching store exceptions and figuring out whether master status has been lost or its an unrelated store error. e.g. An error saying the store has disallowed access vs and error saying the store is unavailable. The first would be an HA exception meaning the RM should stop vs the second exception would mean a loss of store which may be tolerable when storing non-critical information like node health. So, I would like to narrow down from Exception into a better subset as we progress instead of making everything look like an IOException now. Good point. I will mark stuff as Unstable instead for both this and YARN-230
          Hide
          Tom White added a comment -

          I was assuming that the store was HA since it would be backed by HDFS or ZK.

          Anyway, I'm fine refining the set of exceptions as this changes in follow-on JIRAs, even though it probably is just IOException at this point.

          Show
          Tom White added a comment - I was assuming that the store was HA since it would be backed by HDFS or ZK. Anyway, I'm fine refining the set of exceptions as this changes in follow-on JIRAs, even though it probably is just IOException at this point.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12554352/YARN-231.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/169//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554352/YARN-231.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/169//console This message is automatically generated.
          Hide
          Bikas Saha added a comment -

          Attaching patch after rebasing on YARN-230. Addressing review comments and adding default store when recovery is enabled.

          Show
          Bikas Saha added a comment - Attaching patch after rebasing on YARN-230 . Addressing review comments and adding default store when recovery is enabled.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12562132/YARN-231.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/248//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/248//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/248//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562132/YARN-231.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/248//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/248//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/248//console This message is automatically generated.
          Hide
          Bikas Saha added a comment -
          Code	Warning
          IS	Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.zkClient; locked 81% of time
          IS	Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.zkSessionTimeout; locked 50% of time
          

          These are the warnings. I looked at the code and I dont see a synchronization issue. Maybe a different pair of eyes might spot something. If its clean I will disable the warning for that part of the code.

          Show
          Bikas Saha added a comment - Code Warning IS Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.zkClient; locked 81% of time IS Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.zkSessionTimeout; locked 50% of time These are the warnings. I looked at the code and I dont see a synchronization issue. Maybe a different pair of eyes might spot something. If its clean I will disable the warning for that part of the code.
          Hide
          Arun C Murthy added a comment -

          Bikas Saha Apologies for taking this long to respond, but it looks great.

          I'd really want to get this into 2.0.3-alpha to go with YARN-230, but I'm worried about the ZK changes (and since I'm not an expert I can't go very deep into them either).

          So, as a compromise, would you mind splitting this up into 2 patches? a) FS Store b) ZK Store.

          This way I can commit the FS Store into 2.0.3 and then commit ZK Store into trunk for a subsequent release? I know it's being unfair given I've take so long to get around to this, but I'd really appreciate if you could accommodate my request for 2.0.3. Thanks!

          Show
          Arun C Murthy added a comment - Bikas Saha Apologies for taking this long to respond, but it looks great. I'd really want to get this into 2.0.3-alpha to go with YARN-230 , but I'm worried about the ZK changes (and since I'm not an expert I can't go very deep into them either). So, as a compromise, would you mind splitting this up into 2 patches? a) FS Store b) ZK Store. This way I can commit the FS Store into 2.0.3 and then commit ZK Store into trunk for a subsequent release? I know it's being unfair given I've take so long to get around to this, but I'd really appreciate if you could accommodate my request for 2.0.3. Thanks!
          Hide
          Bikas Saha added a comment -

          Attaching a patch containing only FS store. Will upload one for ZK once this goes in.

          Show
          Bikas Saha added a comment - Attaching a patch containing only FS store. Will upload one for ZK once this goes in.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565844/YARN-231.3.FS.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/360//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/360//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565844/YARN-231.3.FS.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/360//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/360//console This message is automatically generated.
          Hide
          Hitesh Shah added a comment -

          @Bikas, regarding the findbug warnings:

          private void createConnection() - not synchronized for zkClient?
          private abstract class ZKAction<T> uses zkSessionTimeout which is synchronized in getNewZooKeeper()

          Show
          Hitesh Shah added a comment - @Bikas, regarding the findbug warnings: private void createConnection() - not synchronized for zkClient? private abstract class ZKAction<T> uses zkSessionTimeout which is synchronized in getNewZooKeeper()
          Hide
          Arun C Murthy added a comment -

          +1, thanks Bikas!

          Show
          Arun C Murthy added a comment - +1, thanks Bikas!
          Hide
          Bikas Saha added a comment -

          Hitesh, createConnection() is private and called from within synchronized functions. ZKAction are invoked from within synchronized functions. Let me know if you see that not happening.

          Show
          Bikas Saha added a comment - Hitesh, createConnection() is private and called from within synchronized functions. ZKAction are invoked from within synchronized functions. Let me know if you see that not happening.
          Hide
          Hitesh Shah added a comment -

          A couple of minor comments on the fs patch:

          FSRMStateStore class:

          • needs a default value for fsWorkingPath in case YarnConfiguration.FS_RM_STATE_STORE_URI is not defined
          • should "FileStatus[] childNodes = fs.listStatus(fsRootDirPath);" use the listStatus function that accepts a path filter?
          Show
          Hitesh Shah added a comment - A couple of minor comments on the fs patch: FSRMStateStore class: needs a default value for fsWorkingPath in case YarnConfiguration.FS_RM_STATE_STORE_URI is not defined should "FileStatus[] childNodes = fs.listStatus(fsRootDirPath);" use the listStatus function that accepts a path filter?
          Hide
          Bikas Saha added a comment -

          There is a default value in yarn-defaults.xml

          +    <name>yarn.resourcemanager.fs.rm-state-store.uri</name>
          +    <value>${hadoop.tmp.dir}/yarn/system/rmstore</value>
          

          Not quite sure about the listStatus API

          Show
          Bikas Saha added a comment - There is a default value in yarn-defaults.xml + <name>yarn.resourcemanager.fs.rm-state-store.uri</name> + <value>${hadoop.tmp.dir}/yarn/system/rmstore</value> Not quite sure about the listStatus API
          Hide
          Hitesh Shah added a comment -

          @Bikas, my mistake - missed the yarn-default change. listStatus with a filter may probably be overkill for the time being. Can ignore that.

          Show
          Hitesh Shah added a comment - @Bikas, my mistake - missed the yarn-default change. listStatus with a filter may probably be overkill for the time being. Can ignore that.
          Hide
          Hitesh Shah added a comment -

          Thanks for the patch, Bikas. Committed to trunk and branch-2.

          Show
          Hitesh Shah added a comment - Thanks for the patch, Bikas. Committed to trunk and branch-2.
          Hide
          Hitesh Shah added a comment -

          FYI, filed YARN-353 for ZK based implementation.

          Show
          Hitesh Shah added a comment - FYI, filed YARN-353 for ZK based implementation.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3272 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3272/)
          YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245)

          Result = SUCCESS
          hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3272 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3272/ ) YARN-231 . RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #105 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/105/)
          YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245)

          Result = SUCCESS
          hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #105 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/105/ ) YARN-231 . RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1294 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1294/)
          YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245)

          Result = FAILURE
          hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1294 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1294/ ) YARN-231 . RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245) Result = FAILURE hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1322 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1322/)
          YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245)

          Result = SUCCESS
          hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1322 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1322/ ) YARN-231 . RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha (Revision 1437245) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437245 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestRMStateStore.java

            People

            • Assignee:
              Bikas Saha
              Reporter:
              Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development