Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3328

SimpleAclAuthorizer can lose ACLs with frequent add/remove calls

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently when adding or removing an ACL with the SimpleAclAuthorizer the following high level steps happen:

      1. read acls from cache
      2. merge with the changes acls
      3. update zookeeper
      4. add a change notification

      Then the Authorizers listening for the change notification know to invalidate their cache and get the latest value. However that takes some time. In the time between the ACL change and the cache update, a new add or remove request could be made. This will follow the steps listed above, and if the cache is not correct all changes from the previous request are lost.

      This can be solved on a single node, by updating the cache at the same time you update zookeeper any time a change is made. However, because there can be multiple instances of the Authorizer, a request could come to a separate authorizer and overwrite the Zookeeper state again loosing changes from earlier requests.

      To solve this on multiple instances. The authorizer could always read/write state from zookeeper (instead of the cache) for add/remove requests and only leverage the cache for get/authorize requests. Or it could block until all the live instances have updated their cache.

      Below is a log from a failed test in the WIP pull request for KAFKA-3266 that shows this behavior:

      [2016-03-03 11:09:20,714] DEBUG [KafkaApi-0] adding User:ANONYMOUS has Allow permission for operations: Describe from hosts: * for Cluster:kafka-cluster (kafka.server.KafkaApis:52)
      [2016-03-03 11:09:20,726] DEBUG updatedAcls: Set(User:ANONYMOUS has Allow permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,738] DEBUG [KafkaApi-0] adding User:ANONYMOUS has Deny permission for operations: Describe from hosts: * for Cluster:kafka-cluster (kafka.server.KafkaApis:52)
      [2016-03-03 11:09:20,739] DEBUG updatedAcls: Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,752] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,755] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,762] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,768] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,773] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      [2016-03-03 11:09:20,777] DEBUG Processing ACL change notification for Cluster:kafka-cluster and Set(User:ANONYMOUS has Deny permission for operations: Describe from hosts: *) (kafka.security.auth.SimpleAclAuthorizer:52)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                granthenke Grant Henke
                Reporter:
                granthenke Grant Henke
                Reviewer:
                Flavio Junqueira
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: