[METRON-1187] Indexing/Profiler Kafka ACL Groups Not Setup Correctly

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Affects Version/s: 0.4.1
• Fix Version/s: 0.4.2
• Assignee: Nick Allen
• Reporter: Nick Allen
• Created: 14/Sep/17 19:44
• Updated: 15/Dec/17 16:56
• Resolved: 15/Dec/17 16:56

Description

When kerberizing Metron using the MPack, either the Profiler or the Indexing Kafka ACL groups will not authorize the 'metron' user. This will only work correctly for the component which is executed first.

This can lead to an error in either the Profiler or Indexing topology that looks like the following.

2017-09-14 07:29:52.984 o.a.s.util [ERROR] Async loop died!
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: indexing

Manually checking confirms the problem.

[root@XXX ambari-server]# /usr/hdp/current/kafka-broker/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=${ZOOKEEPER} --list
[2017-09-14 12:09:26,284] WARN read null data from /kafka-acl-changes/acl_changes_0000000004 when processing notification acl_changes_0000000004 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,304] WARN read null data from /kafka-acl-changes/acl_changes_0000000005 when processing notification acl_changes_0000000005 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,315] WARN read null data from /kafka-acl-changes/acl_changes_0000000006 when processing notification acl_changes_0000000006 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,321] WARN read null data from /kafka-acl-changes/acl_changes_0000000007 when processing notification acl_changes_0000000007 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,323] WARN read null data from /kafka-acl-changes/acl_changes_0000000008 when processing notification acl_changes_0000000008 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,325] WARN read null data from /kafka-acl-changes/acl_changes_0000000009 when processing notification acl_changes_0000000009 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,327] WARN read null data from /kafka-acl-changes/acl_changes_0000000010 when processing notification acl_changes_0000000010 (kafka.common.ZkNodeChangeNotificationListener)
[2017-09-14 12:09:26,337] WARN read null data from /kafka-acl-changes/acl_changes_0000000011 when processing notification acl_changes_0000000011 (kafka.common.ZkNodeChangeNotificationListener)
Current ACLs for resource `Group:bro_parser`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:ambari_kafka_service_check`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Group:profiler`:
 	User:metron has Allow permission for operations: All from hosts: *

[2017-09-14 12:09:26,349] WARN read null data from /kafka-acl-changes/acl_changes_0000000012 when processing notification acl_changes_0000000012 (kafka.common.ZkNodeChangeNotificationListener)
Current ACLs for resource `Group:metron-rest`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:enrichments`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:snort`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:yaf`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Group:enrichments`:
 	User:metron has Allow permission for operations: All from hosts: *

[2017-09-14 12:09:26,351] WARN read null data from /kafka-acl-changes/acl_changes_0000000013 when processing notification acl_changes_0000000013 (kafka.common.ZkNodeChangeNotificationListener)
Current ACLs for resource `Topic:__consumer_offsets`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:bro`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:escalation`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Group:yaf_parser`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Group:snort_parser`:
 	User:metron has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:indexing`:
 	User:metron has Allow permission for operations: All from hosts: *

Issue Links

• links to: GitHub Pull Request #759