Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-149 [Umbrella] ResourceManager (RM) Fail-over
  3. YARN-3006

Improve the error message when attempting manual failover with auto-failover enabled

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.1, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      When executing manual failover with automatic failover enabled, UnsupportedOperationException is thrown.

      # yarn rmadmin -failover rm1 rm2
      Exception in thread "main" java.lang.UnsupportedOperationException: RMHAServiceTarget doesn't have a corresponding ZKFC address
      	at org.apache.hadoop.yarn.client.RMHAServiceTarget.getZKFCAddress(RMHAServiceTarget.java:51)
      	at org.apache.hadoop.ha.HAServiceTarget.getZKFCProxy(HAServiceTarget.java:94)
      	at org.apache.hadoop.ha.HAAdmin.gracefulFailoverThroughZKFCs(HAAdmin.java:311)
      	at org.apache.hadoop.ha.HAAdmin.failover(HAAdmin.java:282)
      	at org.apache.hadoop.ha.HAAdmin.runCmd(HAAdmin.java:449)
      	at org.apache.hadoop.ha.HAAdmin.run(HAAdmin.java:378)
      	at org.apache.hadoop.yarn.client.cli.RMAdminCLI.run(RMAdminCLI.java:482)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
      	at org.apache.hadoop.yarn.client.cli.RMAdminCLI.main(RMAdminCLI.java:622)
      

      I'm thinking the above message is confusing to users. (Users may think whether ZKFC is configured correctly...) The command should output error message to stderr instead of throwing Exception.

      1. YARN-3006.001.patch
        0.9 kB
        Akira Ajisaka

        Issue Links

          Activity

          Hide
          iwasakims Masatake Iwasaki added a comment -

          Is YARN-2807 relevant?

          Show
          iwasakims Masatake Iwasaki added a comment - Is YARN-2807 relevant?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          --forcemanual option is no-op in failover command, so YARN-2807 looks unrelated.
          When attempting failover master node with automatic failover enabled,

          • NameNode: Fail if --forcefence or --forceactive option is set. If these options are not set, do graceful failover.
          • ResourceManager: Always throw UnsupportedOperationException.
          Show
          ajisakaa Akira Ajisaka added a comment - --forcemanual option is no-op in failover command, so YARN-2807 looks unrelated. When attempting failover master node with automatic failover enabled, NameNode: Fail if --forcefence or --forceactive option is set. If these options are not set, do graceful failover. ResourceManager: Always throw UnsupportedOperationException.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Attaching a simple patch.

          Show
          ajisakaa Akira Ajisaka added a comment - Attaching a simple patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12690518/YARN-3006.001.patch
          against trunk revision 788ee35.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6267//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6267//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12690518/YARN-3006.001.patch against trunk revision 788ee35. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6267//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6267//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12690518/YARN-3006.001.patch
          against trunk revision af9d4fe.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7285//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7285//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12690518/YARN-3006.001.patch against trunk revision af9d4fe. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7285//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7285//console This message is automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          +1 for latest patch, committing.

          Show
          leftnoteasy Wangda Tan added a comment - +1 for latest patch, committing.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to trunk/branch-2/branch-2.7, thanks Akira Ajisaka.

          Show
          leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2/branch-2.7, thanks Akira Ajisaka .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #7716 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7716/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7716 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7716/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/181/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/181/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #915 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/915/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #915 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/915/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #172 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/172/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #172 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/172/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2113 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2113/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2113 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2113/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #182 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/182/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #182 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/182/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2131 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2131/)
          YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2131 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2131/ ) YARN-3006 . Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev 7d46a806e71de6692cd230e64e7de18a8252019d) hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java

            People

            • Assignee:
              ajisakaa Akira Ajisaka
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development