[RANGER-3001] Update Ranger KafkaClient to use Kafka AdminClient API instead of Zookeeper

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 3.0.0, 2.2.0
• Component/s: Ranger
• Assignee: Ramesh Mani
• Reporter: Ramesh Mani
• Created: 17/Sep/20 02:12
• Updated: 24/Aug/21 23:39
• Resolved: 12/Oct/20 17:05

Description

Update Ranger KafkaClient to use Kafka AdminClient API instead of Zookeeper.

This would have the following benefits:

• Security: all Kafka commands are channeled and authorized through the broker instead of direct Zookeeper data manipulation. When topics are generated via Zookeeper, it might happen that even Kafka cannot access to the metadata due to ZK ACLs.
• Simplicity/maintainability: the interface of Kafka becomes smaller and there is no need to expose Kafka's own Zookeeper quorum but kept as an implementation detail for storing metadata.
• Compatibility: zookeeper in Apache Kafka will be removed in a future version.

Issue Links

• links to: Review Request