Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4087

Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.2, 2.6.2, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Set YARN_FAIL_FAST to be false by default. If HA is enabled and if there's any state-store error, after the retry operation failed, we always transition RM to standby state.

      Description

      Several fixes:
      1. Set YARN_FAIL_FAST to be false by default, since this makes more sense in production environment.
      2. If HA is enabled and if there's any state-store error, after the retry operation failed, we always transition RM to standby state. Otherwise, we may see two active RMs running. YARN-4107 is one example.

      1. YARN-4087-branch-2.6.patch
        4 kB
        Xuan Gong
      2. YARN-4087.7.patch
        4 kB
        Xuan Gong
      3. YARN-4087.6.patch
        4 kB
        Jian He
      4. YARN-4087.5.patch
        14 kB
        Jian He
      5. YARN-4087.3.patch
        8 kB
        Jian He
      6. YARN-4087.2.patch
        2 kB
        Jian He
      7. YARN-4087.1.patch
        1 kB
        Jian He

        Issue Links

          Activity

          Hide
          jianhe Jian He added a comment -

          simple patch which flips the config

          Show
          jianhe Jian He added a comment - simple patch which flips the config
          Hide
          kasha Karthik Kambatla added a comment -

          +1, if fail-fast hasn't been in any prior release and we are not drastically altering the behavior.

          In any case, it would be nice to release note this new behavior for 2.8.0.

          Show
          kasha Karthik Kambatla added a comment - +1, if fail-fast hasn't been in any prior release and we are not drastically altering the behavior. In any case, it would be nice to release note this new behavior for 2.8.0.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          So by default in yarn-default.xml

          yarn.resourcemanager.fail-fast=true
          yarn.fail-fast=false

          In YarnConfiguration

            public static boolean shouldRMFailFast(Configuration conf) {
              return conf.getBoolean(YarnConfiguration.RM_FAIL_FAST,
                  conf.getBoolean(YarnConfiguration.YARN_FAIL_FAST,
                      YarnConfiguration.DEFAULT_YARN_FAIL_FAST));
            }
          

          some mismatch rt?

          No plans to change YarnConfiguration.RM_FAIL_FAST.

          Show
          bibinchundatt Bibin A Chundatt added a comment - So by default in yarn-default.xml yarn.resourcemanager.fail-fast=true yarn.fail-fast=false In YarnConfiguration public static boolean shouldRMFailFast(Configuration conf) { return conf.getBoolean(YarnConfiguration.RM_FAIL_FAST, conf.getBoolean(YarnConfiguration.YARN_FAIL_FAST, YarnConfiguration.DEFAULT_YARN_FAIL_FAST)); } some mismatch rt? No plans to change YarnConfiguration.RM_FAIL_FAST.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 27s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 2s There were no new javac warning messages.
          +1 javadoc 10m 9s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 58s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 10s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.
              46m 40s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12752615/YARN-4087.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f44b599
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8922/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8922/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8922/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8922/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 2s There were no new javac warning messages. +1 javadoc 10m 9s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 58s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 10s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.     46m 40s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752615/YARN-4087.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f44b599 hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8922/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8922/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8922/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8922/console This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          Bibin A Chundatt, the logic is that default value for RM_FAIL_FAST is YARN_FAIL_FAST

          Show
          jianhe Jian He added a comment - Bibin A Chundatt , the logic is that default value for RM_FAIL_FAST is YARN_FAIL_FAST
          Hide
          hitesh Hitesh Shah added a comment -

          It would be good to rename the config property to something that provides a bit more clarity on what the config knob is meant to control.

          Show
          hitesh Hitesh Shah added a comment - It would be good to rename the config property to something that provides a bit more clarity on what the config knob is meant to control.
          Hide
          jianhe Jian He added a comment -

          The YARN_FAIL_FAST is a global knob to control all components, e.g. RM, NM; The config description does the clarification. Just can't think of a concise and meaningful name. Any naming suggestion is welcome.

          Update the patch to carify the config description more.

          Show
          jianhe Jian He added a comment - The YARN_FAIL_FAST is a global knob to control all components, e.g. RM, NM; The config description does the clarification. Just can't think of a concise and meaningful name. Any naming suggestion is welcome. Update the patch to carify the config description more.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 23s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 51s There were no new javac warning messages.
          +1 javadoc 10m 3s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 0s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.
              46m 22s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12752862/YARN-4087.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / a9c8ea7
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8932/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8932/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8932/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8932/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 23s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 0s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.     46m 22s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752862/YARN-4087.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a9c8ea7 hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8932/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8932/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8932/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8932/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Patch LGTM.

          +1, if fail-fast hasn't been in any prior release and we are not drastically altering the behavior.

          I believe fail-fast just involve recently. However, the default behavior when RM/NM state store get failed could be different from previous releases: it failed NM/RM daemons previously, now we could tolerant it keep running with log some error messages. We should definitely note this in our release notes. Also, may be we should mark this JIRA as incompatible (for behavior)?

          Show
          djp Junping Du added a comment - Patch LGTM. +1, if fail-fast hasn't been in any prior release and we are not drastically altering the behavior. I believe fail-fast just involve recently. However, the default behavior when RM/NM state store get failed could be different from previous releases: it failed NM/RM daemons previously, now we could tolerant it keep running with log some error messages. We should definitely note this in our release notes. Also, may be we should mark this JIRA as incompatible (for behavior)?
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          In yarn-default.xml the default value for RM_FAIL_FAST is true.
          In code the default value for RM_FAIL_FAST is taken from YARN_FAIL_FAST whose value is false.

          Show
          bibinchundatt Bibin A Chundatt added a comment - In yarn-default.xml the default value for RM_FAIL_FAST is true. In code the default value for RM_FAIL_FAST is taken from YARN_FAIL_FAST whose value is false.
          Hide
          jianhe Jian He added a comment -

          In yarn-default.xml the default value for RM_FAIL_FAST is true.

          DIdn't get you. Isn't the default value set to YARN_FAIL_FAST too?

          Show
          jianhe Jian He added a comment - In yarn-default.xml the default value for RM_FAIL_FAST is true. DIdn't get you. Isn't the default value set to YARN_FAIL_FAST too?
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Yes, I just checked that YARN-2019 added the config only in 2.8 which is unreleased now. So, we can safely change the default.

          Also, may be we should mark this JIRA as incompatible (for behavior)?

          The previous behavior was undesired, and nobody in practice should depend on it.

          I think there was a bigger thing that got missed at YARN-2019. If we ignore the failure when the config is off, the higher order operations are stuck in a weird state as there are no retries or explicit app-failures, Jian He?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Yes, I just checked that YARN-2019 added the config only in 2.8 which is unreleased now. So, we can safely change the default. Also, may be we should mark this JIRA as incompatible (for behavior)? The previous behavior was undesired, and nobody in practice should depend on it. I think there was a bigger thing that got missed at YARN-2019 . If we ignore the failure when the config is off, the higher order operations are stuck in a weird state as there are no retries or explicit app-failures, Jian He ?
          Hide
          jianhe Jian He added a comment -

          as there are no retries or explicit app-failures

          Retry already happened internally before the final Exception is thrown.
          Right, app will be stuck at certain state, since no notification is sent back. But, explicitly failing the app may be too harsh, since the app itself can actually proceed without any impact. I think we can still notify back that the store operation is done and let the app continue. Also, print warning message on application page something like "Application is not persisted in state-store due to state-store error. Application will be lost if RM restarted."

          Show
          jianhe Jian He added a comment - as there are no retries or explicit app-failures Retry already happened internally before the final Exception is thrown. Right, app will be stuck at certain state, since no notification is sent back. But, explicitly failing the app may be too harsh, since the app itself can actually proceed without any impact. I think we can still notify back that the store operation is done and let the app continue. Also, print warning message on application page something like "Application is not persisted in state-store due to state-store error. Application will be lost if RM restarted."
          Hide
          adhoot Anubhav Dhoot added a comment -

          In general if we are not failing the daemon if fail fast flag is false, we still need to ensure we are not leaving inconsistent state in RM. For eg in YARN-4032. YARN-2019 is the other case where we did not need to do anything. This would mean every patch from now on that uses fail fast to not crash the daemon should consider taking corrective action to ensure correctness. Does that make sense?

          Show
          adhoot Anubhav Dhoot added a comment - In general if we are not failing the daemon if fail fast flag is false, we still need to ensure we are not leaving inconsistent state in RM. For eg in YARN-4032 . YARN-2019 is the other case where we did not need to do anything. This would mean every patch from now on that uses fail fast to not crash the daemon should consider taking corrective action to ensure correctness. Does that make sense?
          Hide
          jianhe Jian He added a comment -

          Right, we should make sure the state is correct if the error is ignored.

          Uploaded a new patch which addressed Vinod's comment about the inconsistent state.
          Also updated the description about the fixes included in this jira.

          Show
          jianhe Jian He added a comment - Right, we should make sure the state is correct if the error is ignored. Uploaded a new patch which addressed Vinod's comment about the inconsistent state. Also updated the description about the fixes included in this jira.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 9m 53s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 57s The applied patch generated 4 new checkstyle issues (total was 58, now 58).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 34s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common.
          +1 yarn tests 54m 14s Tests passed in hadoop-yarn-server-resourcemanager.
              105m 38s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754094/YARN-4087.3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / ed78b14
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9003/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9003/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 9m 53s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 57s The applied patch generated 4 new checkstyle issues (total was 58, now 58). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 34s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common. +1 yarn tests 54m 14s Tests passed in hadoop-yarn-server-resourcemanager.     105m 38s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754094/YARN-4087.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ed78b14 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9003/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9003/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9003/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 55s There were no new javac warning messages.
          +1 javadoc 10m 13s There were no new javadoc warning messages.
          +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 55s The applied patch generated 6 new checkstyle issues (total was 297, now 299).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          -1 yarn tests 54m 30s Tests failed in hadoop-yarn-server-resourcemanager.
              104m 56s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestRMRestart



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754127/YARN-4087.4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c83d13c
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9005/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9005/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 10m 13s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 55s The applied patch generated 6 new checkstyle issues (total was 297, now 299). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. -1 yarn tests 54m 30s Tests failed in hadoop-yarn-server-resourcemanager.     104m 56s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754127/YARN-4087.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c83d13c checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9005/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9005/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9005/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 9m 49s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 46s The applied patch generated 2 new checkstyle issues (total was 297, now 297).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          -1 yarn tests 52m 9s Tests failed in hadoop-yarn-server-resourcemanager.
              101m 6s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754156/YARN-4087.5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c83d13c
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9008/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9008/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 46s The applied patch generated 2 new checkstyle issues (total was 297, now 297). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. -1 yarn tests 52m 9s Tests failed in hadoop-yarn-server-resourcemanager.     101m 6s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754156/YARN-4087.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c83d13c checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9008/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9008/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9008/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Hi Jian He, thanks for the patch! Can you confirm the test failure is not related to your patch?

          Show
          djp Junping Du added a comment - Hi Jian He , thanks for the patch! Can you confirm the test failure is not related to your patch?
          Hide
          jianhe Jian He added a comment -

          thanks Junping ! I ran the timeout test locally, passing fine.

          Show
          jianhe Jian He added a comment - thanks Junping ! I ran the timeout test locally, passing fine.
          Hide
          jianhe Jian He added a comment -

          Had some offline discussion with Vinod and Xuan, simply ignoring failures and let app continue is also not good, since app will be lost after restart.
          Uploaded a new patch which removes this part of change. I'll open a new jira regarding how to handle inconsistent state if ZK is unavailable.

          Show
          jianhe Jian He added a comment - Had some offline discussion with Vinod and Xuan, simply ignoring failures and let app continue is also not good, since app will be lost after restart. Uploaded a new patch which removes this part of change. I'll open a new jira regarding how to handle inconsistent state if ZK is unavailable.
          Hide
          xgong Xuan Gong added a comment -

          +1 For the Latest patch.
          Junping Du Do you have any other comments ?

          Show
          xgong Xuan Gong added a comment - +1 For the Latest patch. Junping Du Do you have any other comments ?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 49s There were no new javac warning messages.
          +1 javadoc 10m 6s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 50s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
          +1 yarn tests 54m 28s Tests passed in hadoop-yarn-server-resourcemanager.
              104m 41s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754308/YARN-4087.6.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / bcc85e3
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9015/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9015/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 10m 6s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 50s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 54m 28s Tests passed in hadoop-yarn-server-resourcemanager.     104m 41s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754308/YARN-4087.6.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / bcc85e3 hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9015/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9015/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9015/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks Jian He for updating the patch and Xuan Gong for the review! Patch LGTM mostly, except one NIT here:

          +      LOG.warn("Fail RM now due to state-store error!");
          

          I think we should use error or even fatal for this message.

          Show
          djp Junping Du added a comment - Thanks Jian He for updating the patch and Xuan Gong for the review! Patch LGTM mostly, except one NIT here: + LOG.warn( "Fail RM now due to state-store error!" ); I think we should use error or even fatal for this message.
          Hide
          xgong Xuan Gong added a comment -

          Thanks Junping Du for the review. Created a new patch to address this comment.

          Show
          xgong Xuan Gong added a comment - Thanks Junping Du for the review. Created a new patch to address this comment.
          Hide
          djp Junping Du added a comment -

          +1. v7 patch LGTM.

          Show
          djp Junping Du added a comment - +1. v7 patch LGTM.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 18m 9s Findbugs (version 3.0.0) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 54s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 22s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 37s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          +1 yarn tests 53m 57s Tests passed in hadoop-yarn-server-resourcemanager.
              101m 28s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754541/YARN-4087.7.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1dbd8e3
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9025/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9025/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 9s Findbugs (version 3.0.0) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 54s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 37s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. +1 yarn tests 53m 57s Tests passed in hadoop-yarn-server-resourcemanager.     101m 28s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754541/YARN-4087.7.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1dbd8e3 hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9025/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9025/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9025/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8411 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8411/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8411 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8411/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2/branch-2.7/branch-2.6.
          Thanks, Jian He for the patch, and Junping Du for the review

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2/branch-2.7/branch-2.6. Thanks, Jian He for the patch, and Junping Du for the review
          Hide
          bikassaha Bikas Saha added a comment -

          Repeating my comments from YARN-2019 here
          There would be 2 kinds of state store operations - reads and writes. Writes may be of 2 kinds - critical and non-critical. E.g. saving an application submission is critical. Saving a node information is perhaps not critical. It would affect system correctness is critical write operation errors are allowed to be ignored. We end up with YARN-4118 and other such potential issues. Essentially we are turning a write-ahead log into something that optional. I dont see how the system can make stable reliability guarantees by making the write-ahead log non-fatal.
          On the other hand read errors or non-critical write errors should not block RM progress but do need to be potentially retried. That also does not seem to be addressed in the patch.

          Show
          bikassaha Bikas Saha added a comment - Repeating my comments from YARN-2019 here There would be 2 kinds of state store operations - reads and writes. Writes may be of 2 kinds - critical and non-critical. E.g. saving an application submission is critical. Saving a node information is perhaps not critical. It would affect system correctness is critical write operation errors are allowed to be ignored. We end up with YARN-4118 and other such potential issues. Essentially we are turning a write-ahead log into something that optional. I dont see how the system can make stable reliability guarantees by making the write-ahead log non-fatal. On the other hand read errors or non-critical write errors should not block RM progress but do need to be potentially retried. That also does not seem to be addressed in the patch.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #359 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/359/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #359 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/359/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1090 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1090/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1090 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1090/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #352 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/352/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #352 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/352/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2302 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2302/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2302 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2302/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2280/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2280/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #341 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/341/)
          YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #341 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/341/ ) YARN-4087 . Followup fixes after YARN-2019 regarding RM behavior when (xgong: rev 9b78e6e33d8c117c1e909df414f20d9db56efe4b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          kasha Karthik Kambatla added a comment -

          Marking this incompatible to ensure this gets called out in terms of behavior change.

          Show
          kasha Karthik Kambatla added a comment - Marking this incompatible to ensure this gets called out in terms of behavior change.

            People

            • Assignee:
              jianhe Jian He
              Reporter:
              jianhe Jian He
            • Votes:
              1 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development