ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1634

A new feature proposal to ZooKeeper: authentication enforcement

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.4.5
    • Fix Version/s: 3.6.0
    • Component/s: security, server
    • Labels:
      None
    • Tags:
      authentication

      Description

      Up to the version of 3.4.5, ZooKeeperServer doesn't force the authentication if the client doesn't give any auth-info through ZooKeeper#addAuthInfo method invocation. Hence, every znode should have at least one ACL assigned otherwise any unauthenticated client can do anything on it.

      The current authentication/authorization mechanism of ZooKeeper described above has several points at issue:
      1. At security standpoint, a maleficent client can access a znode which doesn't have any proper authorization access control set.
      2. At runtime performance standpoint, authorization for every znode to every operation is unnecessarily but always evaluated against the client who bypassed the authentication phase.

      In other words, the current mechanism doesn't address a certain requirement at below:
      "We want to protect a ZK server by enforcing a simple authentication to every client no matter which znode it is trying to access. Every connection (or operation) from the client won't be established but rejected if it doesn't come with a valid authentication information. As we don't have any other distinction between znodes in term of authorization, we don't want any ACLs on any znode."

      To address the issues mentioned above, we propose a feature called "authentication enforcement" to the ZK source. The idea is roughly but clearly described in a form of patch in the attached file (zookeeper_3.4.5_patch_for_authentication_enforcement.patch): which makes ZooKeeperServer enforce the authentication with the given 2 configurations: authenticationEnforced (boolean) and enforcedAuthenticationScheme (string) against every operation coming through ZooKeeperServer#processPacket method except for OpCode.auth operation. The repository base of the patch is "http://svn.apache.org/repos/asf/zookeeper/tags/release-3.4.5/"

        Issue Links

          Activity

          Hide
          Jaewoong Choi added a comment -

          This patch is incomplete meaning not directly applicable but I believe it's enough showing the idea of the proposal.

          Show
          Jaewoong Choi added a comment - This patch is incomplete meaning not directly applicable but I believe it's enough showing the idea of the proposal.
          Hide
          Patrick Hunt added a comment -

          Why wouldn't we just implement SSL encryption on the client-server pipes and then use bi-di certs for authentication? There's a long standing patch to add netty support for client-server communication, which would simplify this (adding netty ssl support to that is trivial).

          Show
          Patrick Hunt added a comment - Why wouldn't we just implement SSL encryption on the client-server pipes and then use bi-di certs for authentication? There's a long standing patch to add netty support for client-server communication, which would simplify this (adding netty ssl support to that is trivial).
          Hide
          Alex Guan added a comment -

          I believe the idea is to reuse the the existing authentication framework which is extensible and much more flexible than just providing ssl based authentication.

          Show
          Alex Guan added a comment - I believe the idea is to reuse the the existing authentication framework which is extensible and much more flexible than just providing ssl based authentication.
          Hide
          Jaewoong Choi added a comment -

          SSL is simply just overburden for the given authentication requirement both for service and client sides where there's no any need of channel security for both sides.

          Show
          Jaewoong Choi added a comment - SSL is simply just overburden for the given authentication requirement both for service and client sides where there's no any need of channel security for both sides.
          Hide
          Jaewoong Choi added a comment -

          To clarify, both client and server don't care about chances of eavesdropping or tampering of communication. Assume the situation that communication doesn't include any data should be secured. Only server cares about the authentication of the client so that it can't deny unidentified connection during the authentication phase efficiently in one shot. In this requirement, why would we encrypt all data packets with cost at socket layer? SSL may can be considered but separately only if there comes a need of channel security. Totally different requirement.

          Show
          Jaewoong Choi added a comment - To clarify, both client and server don't care about chances of eavesdropping or tampering of communication. Assume the situation that communication doesn't include any data should be secured. Only server cares about the authentication of the client so that it can't deny unidentified connection during the authentication phase efficiently in one shot. In this requirement, why would we encrypt all data packets with cost at socket layer? SSL may can be considered but separately only if there comes a need of channel security. Totally different requirement.
          Hide
          Andrew Purtell added a comment -

          As much as I'd personally like to see a move away from Kerberos auth to TLS in the Apache ecosystem intersecting and related to Hadoop, things are moving in the other direction. Projects like Kafka which started with TLS are adding krb auth options. Zookeeper is well positioned as is.

          Show
          Andrew Purtell added a comment - As much as I'd personally like to see a move away from Kerberos auth to TLS in the Apache ecosystem intersecting and related to Hadoop, things are moving in the other direction. Projects like Kafka which started with TLS are adding krb auth options. Zookeeper is well positioned as is.
          Hide
          Flavio Junqueira added a comment -

          Andrew Purtell Kafka allows authentication via SSL:

          http://kafka.apache.org/documentation.html#security_ssl

          Although it does support authentication with SASL+Kerberos. I think that also supporting SSL auth like in Kafka would be a nice addition, do you agree?

          Show
          Flavio Junqueira added a comment - Andrew Purtell Kafka allows authentication via SSL: http://kafka.apache.org/documentation.html#security_ssl Although it does support authentication with SASL+Kerberos. I think that also supporting SSL auth like in Kafka would be a nice addition, do you agree?
          Show
          Flavio Junqueira added a comment - Actually, we already support it: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide
          Hide
          Andrew Purtell added a comment -

          You should read my comment as agreeing with other comments here that if "authorization enforcement" didn't work with the current framework, but somehow required TLS, then this would be unsatisfactory for all those environments based on Kerberos

          Show
          Andrew Purtell added a comment - You should read my comment as agreeing with other comments here that if "authorization enforcement" didn't work with the current framework, but somehow required TLS, then this would be unsatisfactory for all those environments based on Kerberos
          Hide
          Andrew Purtell added a comment -

          *authenticaion enforcement

          Show
          Andrew Purtell added a comment - *authenticaion enforcement
          Hide
          ASF GitHub Bot added a comment -

          GitHub user hanm opened a pull request:

          https://github.com/apache/zookeeper/pull/118

          ZOOKEEPER-1634: hardening security by teaching server to enforce client authentication.

            1. Motivation
              Previously ZooKeeper server is open to the world as the server does not enforce client authentication - anonymous clients are allowed to establish session with server. This behavior raises a couple of issues from the perspective of performance and security for example:
          • It is easy to launch a DDoS attack to server, by having a fleet of anonymous clients connect to the ensemble, as each session would consume valuable resources (socket, memory, etc) from server.
          • It is cumbersome to enforce certain security models with the presence of anonymous clients login - for example as clients are not trusted the root ACL has to be disabled for writes to world, among other configurations an admin has to do to secure a cluster in a multi-tenant environment.

          So the goal here is to address such issue by hardening ZooKeeper security to provide a more confined access option that user could opt-in, which in addition to the existing ACLs together could lead to more secured / resource optimal ensemble.

            1. Design Abstract
          • Introduce a new server side Java property that if set, ZooKeeper server will only accept connections and requests from clients that have authenticated with server via SASL.
          • Clients that are not configured with SASL authentication, or configured with SASL but fail authentication (i.e. with invalid credential) will not be able to establish a session with server. A typed error code (-124) will be delivered in such case, both Java and C client will close the session with server thereafter, without further attempts on retrying to reconnect.
          • This feature overrules the server property "zookeeper.allowSaslFailedClients". So even if server is configured to allow clients that fail SASL authentication to login, client will not be able to establish a session with server if this feature is enabled.
          • Only support SASL because only SASL authentication has the property that no operations will happen until SASL authentication process finished. Thus, the decision of whether to close the session or not can be quickly made on server side upon receiving a client connection request. We could later add support for other auth scheme via add_auth_info if that's desired (if we do, then a session has to be maintained until add_auth_info is invoked.).
          • As a side benefit, this PR fixes an issue mentioned in ZOOKEEPER-2346 by correctly propagate events from server to client side so a SASL auth failure will manifest as an auth / config failure rather than generic ConnectionLoss event.

          JIRA: https://issues.apache.org/jira/browse/ZOOKEEPER-1634
          The PR also covers (or part of):
          https://issues.apache.org/jira/browse/ZOOKEEPER-2462
          https://issues.apache.org/jira/browse/ZOOKEEPER-2526
          https://issues.apache.org/jira/browse/ZOOKEEPER-2346

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hanm/zookeeper ZOOKEEPER-1634

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zookeeper/pull/118.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #118


          commit 90beaa0396cb2238b40e4b93764bd1a396b9047b
          Author: Michael Han <hanm@cloudera.com>
          Date: 2016-11-29T23:19:38Z

          ZOOKEEPER-1634: teach server to enforce client authentication.


          Show
          ASF GitHub Bot added a comment - GitHub user hanm opened a pull request: https://github.com/apache/zookeeper/pull/118 ZOOKEEPER-1634 : hardening security by teaching server to enforce client authentication. Motivation Previously ZooKeeper server is open to the world as the server does not enforce client authentication - anonymous clients are allowed to establish session with server. This behavior raises a couple of issues from the perspective of performance and security for example: It is easy to launch a DDoS attack to server, by having a fleet of anonymous clients connect to the ensemble, as each session would consume valuable resources (socket, memory, etc) from server. It is cumbersome to enforce certain security models with the presence of anonymous clients login - for example as clients are not trusted the root ACL has to be disabled for writes to world, among other configurations an admin has to do to secure a cluster in a multi-tenant environment. So the goal here is to address such issue by hardening ZooKeeper security to provide a more confined access option that user could opt-in, which in addition to the existing ACLs together could lead to more secured / resource optimal ensemble. Design Abstract Introduce a new server side Java property that if set, ZooKeeper server will only accept connections and requests from clients that have authenticated with server via SASL. Clients that are not configured with SASL authentication, or configured with SASL but fail authentication (i.e. with invalid credential) will not be able to establish a session with server. A typed error code (-124) will be delivered in such case, both Java and C client will close the session with server thereafter, without further attempts on retrying to reconnect. This feature overrules the server property "zookeeper.allowSaslFailedClients". So even if server is configured to allow clients that fail SASL authentication to login, client will not be able to establish a session with server if this feature is enabled. Only support SASL because only SASL authentication has the property that no operations will happen until SASL authentication process finished. Thus, the decision of whether to close the session or not can be quickly made on server side upon receiving a client connection request. We could later add support for other auth scheme via add_auth_info if that's desired (if we do, then a session has to be maintained until add_auth_info is invoked.). As a side benefit, this PR fixes an issue mentioned in ZOOKEEPER-2346 by correctly propagate events from server to client side so a SASL auth failure will manifest as an auth / config failure rather than generic ConnectionLoss event. JIRA: https://issues.apache.org/jira/browse/ZOOKEEPER-1634 The PR also covers (or part of): https://issues.apache.org/jira/browse/ZOOKEEPER-2462 https://issues.apache.org/jira/browse/ZOOKEEPER-2526 https://issues.apache.org/jira/browse/ZOOKEEPER-2346 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hanm/zookeeper ZOOKEEPER-1634 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/118.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #118 commit 90beaa0396cb2238b40e4b93764bd1a396b9047b Author: Michael Han <hanm@cloudera.com> Date: 2016-11-29T23:19:38Z ZOOKEEPER-1634 : teach server to enforce client authentication.
          Hide
          Hadoop QA added a comment -

          -1 overall. GitHub Pull Request Build

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 38 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 38 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/94//console This message is automatically generated.

            People

            • Assignee:
              Michael Han
              Reporter:
              Jaewoong Choi
            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 72h
                72h
                Remaining:
                Remaining Estimate - 72h
                72h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development