Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0
    • Fix Version/s: 3.0.0, 2.0.3-alpha
    • Component/s: ipc, security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      RPC.Server should always allow digest auth (tokens) if a secret manager if present.

      1. HADOOP-8783.patch
        6 kB
        Daryn Sharp
      2. HADOOP-8783.patch
        9 kB
        Daryn Sharp

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544553/HADOOP-8783.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
        org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
        org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
        org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
        org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
        org.apache.hadoop.hdfs.server.namenode.TestEditLog
        org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
        org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
        org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
        org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer
        org.apache.hadoop.hdfs.TestDFSUpgrade

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1432//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1432//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544553/HADOOP-8783.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits org.apache.hadoop.hdfs.server.namenode.TestEditLog org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer org.apache.hadoop.hdfs.TestDFSUpgrade +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1432//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1432//console This message is automatically generated.
        Hide
        Daryn Sharp added a comment -

        Removing hdfs change accidentally included in patch.

        Show
        Daryn Sharp added a comment - Removing hdfs change accidentally included in patch.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544639/HADOOP-8783.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1437//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1437//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544639/HADOOP-8783.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1437//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1437//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        +1 (non-binding) Looks good to me. I hope better testing will be added with the client-side changes.

        Show
        Kihwal Lee added a comment - +1 (non-binding) Looks good to me. I hope better testing will be added with the client-side changes.
        Hide
        Daryn Sharp added a comment -

        Yes, after the client changes, all combinations of secure/insecure client & server and the resulting auth can be easily tested.

        Show
        Daryn Sharp added a comment - Yes, after the client changes, all combinations of secure/insecure client & server and the resulting auth can be easily tested.
        Hide
        Owen O'Malley added a comment -

        +1 lgtm

        One minor nit is that I'd replace the default case with KERBEROS so that if someone adds a new authentication mechanism that this code would stop compiling rather than do the wrong thing.

        Show
        Owen O'Malley added a comment - +1 lgtm One minor nit is that I'd replace the default case with KERBEROS so that if someone adds a new authentication mechanism that this code would stop compiling rather than do the wrong thing.
        Hide
        Owen O'Malley added a comment -

        Actually, since java doesn't complain if the cases aren't covered, just commit your current patch.

        Show
        Owen O'Malley added a comment - Actually, since java doesn't complain if the cases aren't covered, just commit your current patch.
        Hide
        Daryn Sharp added a comment -

        I'd like guidance on where I should integrate this patch. We're considering it for 23, so that would seem to imply it has to be branch-2. Owen thinks we should consider it for branch-1 as well. My suggestion would be to go into trunk and branch-2, and once all the umbrella changes are in, we can consider 23 and branch-1. Thoughts?

        Show
        Daryn Sharp added a comment - I'd like guidance on where I should integrate this patch. We're considering it for 23, so that would seem to imply it has to be branch-2. Owen thinks we should consider it for branch-1 as well. My suggestion would be to go into trunk and branch-2, and once all the umbrella changes are in, we can consider 23 and branch-1. Thoughts?
        Hide
        Eli Collins added a comment -

        What's the impact? Looks like HADOOP-8779 is slated for branch-2 so I was assuming all the subtasks would go in branch-2 as well.

        Show
        Eli Collins added a comment - What's the impact? Looks like HADOOP-8779 is slated for branch-2 so I was assuming all the subtasks would go in branch-2 as well.
        Hide
        Daryn Sharp added a comment -

        I guess this discussion is really about the target releases for HADOOP-8779. Overall, the parent jira will make the "security enabled" code paths always be activated. We know these work, so having tokens always enabled reduces code paths, increases code coverage, and enables thorough testing of tokens via pre-commit tests, etc.

        Show
        Daryn Sharp added a comment - I guess this discussion is really about the target releases for HADOOP-8779 . Overall, the parent jira will make the "security enabled" code paths always be activated. We know these work, so having tokens always enabled reduces code paths, increases code coverage, and enables thorough testing of tokens via pre-commit tests, etc.
        Hide
        Eli Collins added a comment -

        +1 to branch-2

        Show
        Eli Collins added a comment - +1 to branch-2
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2805 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2805/)
        HADOOP-8783. Improve RPC.Server's digest auth (daryn) (Revision 1393483)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2805 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2805/ ) HADOOP-8783 . Improve RPC.Server's digest auth (daryn) (Revision 1393483) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2867 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2867/)
        HADOOP-8783. Improve RPC.Server's digest auth (daryn) (Revision 1393483)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2867 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2867/ ) HADOOP-8783 . Improve RPC.Server's digest auth (daryn) (Revision 1393483) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Hide
        Daryn Sharp added a comment -

        Will use umbrella jira to later track integration into 23.x and possibly 1.x

        Show
        Daryn Sharp added a comment - Will use umbrella jira to later track integration into 23.x and possibly 1.x
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2828 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2828/)
        HADOOP-8783. Improve RPC.Server's digest auth (daryn) (Revision 1393483)

        Result = FAILURE
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2828 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2828/ ) HADOOP-8783 . Improve RPC.Server's digest auth (daryn) (Revision 1393483) Result = FAILURE daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/)
        HADOOP-8783. Improve RPC.Server's digest auth (daryn) (Revision 1393483)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/ ) HADOOP-8783 . Improve RPC.Server's digest auth (daryn) (Revision 1393483) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/)
        HADOOP-8783. Improve RPC.Server's digest auth (daryn) (Revision 1393483)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/ ) HADOOP-8783 . Improve RPC.Server's digest auth (daryn) (Revision 1393483) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393483 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development