Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3284

bootstrapStandby fails in secure cluster

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: ha, security
    • Labels:
      None

      Description

      HDFS-3247 improved bootstrapStandby to check if the other NN is in active state before trying to bootstrap. But, it forgot to set up the kerberos principals in the config before doing so. So, bootstrapStandby now fails with "Failed to specify server's Kerberos principal name" in a secure cluster. (Credit to Stephen Chu for finding this)

      1. hdfs-3284.txt
        4 kB
        Todd Lipcon
      2. hdfs-3284.txt
        2 kB
        Todd Lipcon

        Activity

        Todd Lipcon created issue -
        Hide
        Todd Lipcon added a comment -

        Simple patch to fix this. I tested manually on a secure cluster and verified the problem is fixed.

        No unit tests because I can't get the new secure test infrastructure to actually work as advertised.

        Show
        Todd Lipcon added a comment - Simple patch to fix this. I tested manually on a secure cluster and verified the problem is fixed. No unit tests because I can't get the new secure test infrastructure to actually work as advertised.
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment hdfs-3284.txt [ 12522829 ]
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me.

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me.
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522829/hdfs-3284.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The patch appears to cause tar ant target to fail.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2280//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2280//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522829/hdfs-3284.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2280//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2280//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        ah, the addSecurityConfiguration function is only on the auto-HA branch. Let me pull that into this patch as well.

        Show
        Todd Lipcon added a comment - ah, the addSecurityConfiguration function is only on the auto-HA branch. Let me pull that into this patch as well.
        Hide
        Todd Lipcon added a comment -

        Patch which should work on trunk (re-tested with a secure trunk HA cluster)

        Show
        Todd Lipcon added a comment - Patch which should work on trunk (re-tested with a secure trunk HA cluster)
        Todd Lipcon made changes -
        Attachment hdfs-3284.txt [ 12522849 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522849/hdfs-3284.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2282//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2282//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522849/hdfs-3284.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2282//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2282//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        The test failure is unrelated (this patch doesn't touch that area of the code)

        Show
        Todd Lipcon added a comment - The test failure is unrelated (this patch doesn't touch that area of the code)
        Hide
        Aaron T. Myers added a comment -

        +1, the updated patch looks good to me. Thanks a lot for taking care of this, Todd.

        Show
        Aaron T. Myers added a comment - +1, the updated patch looks good to me. Thanks a lot for taking care of this, Todd.
        Hide
        Todd Lipcon added a comment -

        Committed to trunk and branch-2.

        Show
        Todd Lipcon added a comment - Committed to trunk and branch-2.
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.0.0 [ 12320353 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2082 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2082/)
        HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813)

        Result = SUCCESS
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2082 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2082/ ) HDFS-3284 . bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2155 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2155/)
        HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813)

        Result = SUCCESS
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2155 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2155/ ) HDFS-3284 . bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2096 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2096/)
        HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813)

        Result = ABORTED
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2096 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2096/ ) HDFS-3284 . bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1017 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1017/)
        HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813)

        Result = FAILURE
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1017 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1017/ ) HDFS-3284 . bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1052 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1052/)
        HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813)

        Result = FAILURE
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1052 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1052/ ) HDFS-3284 . bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon. (Revision 1326813) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1326813 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSHAAdmin.java

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development