Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.7.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In some use cases, it make sense to authorize the usage of some services only from specific hosts. Just like ACLS for Service Authorization , there can be a list of hosts for each service and this list can be checked during authorization.

      Similar to ACLS, there can be a whitelist of ip and blacklist of ips. The default whitelist will be * and default blacklist will be empty. It should be possible to override the default whitelist and default blacklist. It should be possible to define whitelist and blacklist per service.
      It should be possible to define ip ranges in blacklists and whitelists

      1. HADOOP-10651.patch
        21 kB
        Benoy Antony
      2. HADOOP-10651.patch
        16 kB
        Benoy Antony
      3. HADOOP-10651.patch
        16 kB
        Benoy Antony

        Issue Links

          Activity

          Hide
          benoyantony Benoy Antony added a comment -

          Attaching the patch which authorizes service access from specific hosts.
          The serviceLevelAuth documentation is updated with the new ability.

          Uses MachineList to check whether the client address is included in the specified hosts.
          MachineList supports specification of hosts via hostnames, ip addresses, and ip ranges (CIDR format).

          Show
          benoyantony Benoy Antony added a comment - Attaching the patch which authorizes service access from specific hosts. The serviceLevelAuth documentation is updated with the new ability. Uses MachineList to check whether the client address is included in the specified hosts. MachineList supports specification of hosts via hostnames, ip addresses, and ip ranges (CIDR format).
          Hide
          benoyantony Benoy Antony added a comment -

          rebased the patch based on HADOOP-10650

          Show
          benoyantony Benoy Antony added a comment - rebased the patch based on HADOOP-10650
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12665544/HADOOP-10651.patch
          against trunk revision 270a271.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4603//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4603//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12665544/HADOOP-10651.patch against trunk revision 270a271. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4603//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4603//console This message is automatically generated.
          Hide
          benoyantony Benoy Antony added a comment -

          Could I please get a review of this feature and patch ?

          Show
          benoyantony Benoy Antony added a comment - Could I please get a review of this feature and patch ?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Benoy Antony,

          The patch looks fine. Couple of comments on ServiceAuthorizationManager.java:

          1. Could you apply the coding style consistently? There are missing and extra whitespace in the added chunks.
          2. For protocolToMachineLists, could you add a comment similar to protocolToAcls. i.e. the first array entry is the set of allowed hosts and the second is the set of blocked hosts.

          Also I think a couple of comments in TestServiceAuthorization.java need to be fixed. e.g. Lines 328 and 335, comments should start with "TestProtocol1...".

          Thank you for updating the documentation.

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Benoy Antony , The patch looks fine. Couple of comments on ServiceAuthorizationManager.java: Could you apply the coding style consistently? There are missing and extra whitespace in the added chunks. For protocolToMachineLists, could you add a comment similar to protocolToAcls. i.e. the first array entry is the set of allowed hosts and the second is the set of blocked hosts. Also I think a couple of comments in TestServiceAuthorization.java need to be fixed. e.g. Lines 328 and 335, comments should start with "TestProtocol1...". Thank you for updating the documentation.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12665544/HADOOP-10651.patch
          against trunk revision 40ee4bf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12665544/HADOOP-10651.patch against trunk revision 40ee4bf. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5345//console This message is automatically generated.
          Hide
          benoyantony Benoy Antony added a comment -

          Attaching the patch which addresses comments by Arpit Agarwal

          Show
          benoyantony Benoy Antony added a comment - Attaching the patch which addresses comments by Arpit Agarwal
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12690645/HADOOP-10651.patch
          against trunk revision e13a484.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The following test timeouts occurred in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverControllerStress

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5378//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5378//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12690645/HADOOP-10651.patch against trunk revision e13a484. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The following test timeouts occurred in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverControllerStress Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5378//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5378//console This message is automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the latest patch.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the latest patch.
          Hide
          benoyantony Benoy Antony added a comment -

          Thanks for the review, Arpit Agarwal.
          If there are no further comments, I'll commit this tomorrow.

          Show
          benoyantony Benoy Antony added a comment - Thanks for the review, Arpit Agarwal . If there are no further comments, I'll commit this tomorrow.
          Hide
          benoyantony Benoy Antony added a comment -

          Committed to trunk and branch-2.

          Show
          benoyantony Benoy Antony added a comment - Committed to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #6831 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6831/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6831 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6831/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #68 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/68/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #68 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/68/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #802 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/802/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #802 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/802/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #65 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/65/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #65 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/65/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2000 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2000/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2000 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2000/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #69 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/69/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #69 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/69/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2019 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2019/)
          HADOOP-10651. Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java
          • hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2019 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2019/ ) HADOOP-10651 . Add ability to restrict service access using IP addresses and hostnames. (Benoy Antony) (benoy: rev 20625c8f048701c9516da159b24c0b33983e4bb7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestServiceAuthorization.java hadoop-common-project/hadoop-common/src/site/apt/ServiceLevelAuth.apt.vm hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java

            People

            • Assignee:
              benoyantony Benoy Antony
              Reporter:
              benoyantony Benoy Antony
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development