Hadoop Common
  1. Hadoop Common
  2. HADOOP-8243

Security support broken in CLI (manual) failover controller

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: ha, security
    • Labels:
      None

      Description

      Some recent refactoring accidentally caused the proxies in some places to get created with a default Configuration, instead of using the Configuration set up by the DFSHAAdmin tool. This causes the HAServiceProtocol to be missing the configuration which specifies the NN principle – and thus breaks the CLI HAAdmin tool in secure setups.

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1040 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1040/)
        HADOOP-8243. Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135)

        Result = FAILURE
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1040 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1040/ ) HADOOP-8243 . Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1005 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1005/)
        HADOOP-8243. Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135)

        Result = FAILURE
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1005 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1005/ ) HADOOP-8243 . Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1997 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1997/)
        HADOOP-8243. Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135)

        Result = SUCCESS
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1997 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1997/ ) HADOOP-8243 . Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1984 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1984/)
        HADOOP-8243. Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135)

        Result = SUCCESS
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1984 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1984/ ) HADOOP-8243 . Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2059 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2059/)
        HADOOP-8243. Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135)

        Result = SUCCESS
        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2059 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2059/ ) HADOOP-8243 . Security support broken in CLI (manual) failover controller. Contributed by Todd Lipcon. (Revision 1309135) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1309135 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAServiceTarget.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
        Todd Lipcon made changes -
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.0.0 [ 12320352 ]
        Hide
        Todd Lipcon added a comment -

        Committed to branch-2 and trunk, thx for the review

        Show
        Todd Lipcon added a comment - Committed to branch-2 and trunk, thx for the review
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me. Thanks a lot, Todd.

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me. Thanks a lot, Todd.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521040/hadoop-8243.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.fs.viewfs.TestViewFsTrash

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/814//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/814//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521040/hadoop-8243.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/814//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/814//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        I should note I also ran TestDFSHAAdmin and TestDFSHAAdminMiniCluster against this common patch, and they both passed.

        Show
        Todd Lipcon added a comment - I should note I also ran TestDFSHAAdmin and TestDFSHAAdminMiniCluster against this common patch, and they both passed.
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment hadoop-8243.txt [ 12521040 ]
        Hide
        Todd Lipcon added a comment -

        This patch fixes the issue. I tested manually by setting up a secure HA cluster and running haadmin with -failover, -checkHealth, -transitionToActive, -transitionToStandby, and -getServiceState subcommands.

        These manual tests are covered by the HA test plan.

        Unfortunately there are no automated tests for security here. I tried to get the new ApacheDS-based security test infrastructure to work, but it doesn't work correctly in my environment. Please see my comment here: https://issues.apache.org/jira/browse/HDFS-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242722#comment-13242722

        Show
        Todd Lipcon added a comment - This patch fixes the issue. I tested manually by setting up a secure HA cluster and running haadmin with -failover , -checkHealth , -transitionToActive , -transitionToStandby , and -getServiceState subcommands. These manual tests are covered by the HA test plan. Unfortunately there are no automated tests for security here. I tried to get the new ApacheDS-based security test infrastructure to work, but it doesn't work correctly in my environment. Please see my comment here: https://issues.apache.org/jira/browse/HDFS-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242722#comment-13242722
        Todd Lipcon created issue -

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development