Details
Major Description
In a kerberized environment with Ranger, Kafka client is unable to list consumer groups to iterate over if the user only has Describe permission on their own topics rather than on all topics.
/usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --bootstrap-server <fqdn> --list
It ends up with a blank output instead of the list of consumer groups.
If you then grant Describe permission to all topics, that command then gives you a list of consumer groups as expected.
I believe Kafka permissions have been improved to be more granular in KAFKA-6058.
Ranger needs to be updated to reflect these more granular Kafka permissions to allow listing consumer groups without having to also have describe permissions to all topics.
Interestingly I can still describe a consumer group after I have revoked my own permissions and agent policy has been updated if I know the name of the consumer group, but it omits the topic for which I no longer have permission.
/usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --bootstrap-server <fqdn> --describe --group <custom>.<custom>