[IMPALA-13112] Remove Kafka Dependency Exclusion from ranger-plugins-audit in Impala FE pom.xml

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 4.1.0, Impala 4.2.0, Impala 4.1.1, Impala 4.1.2, Impala 4.3.0, Impala 4.4.0
• Component/s: fe
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: woosuk.ro
• Created: 29/May/24 07:57
• Updated: 29/May/24 22:27

Description

Background:

Previously, Impala removed the Kafka dependency from the ranger-plugins-audit module due to conflicts with the JAX-RS package. This conflict resulted in errors like java.lang.NoClassDefFoundError: javax/ws/rs/core/Link$Builder. Relevant commits include:

1. Commit a1f4a849cf1c4f51e0acc737c7da166651645870
2. Commit f85dbff97618066d60f37736808c8c24aa0a98e5

These commits addressed the issue by excluding all Kafka dependencies to avoid JAX-RS version conflicts, specifically related to kafka_2.11 and later kafka_2.12.

Current Situation:

Recent tests indicate that the latest version of kafka-clients-2.8.1.jar used by Apache Ranger no longer causes the previous JAX-RS conflicts. Therefore, we propose to remove the exclusion of Kafka dependencies from ranger-plugins-audit in Impala's pom.xml. This change is necessary to utilize Ranger's kafkaAuditProvider, which requires Kafka clients.

Proposal:

Modify the pom.xml file in Impala FE to remove the Kafka dependency exclusion from ranger-plugins-audit. This update will re-enable the Kafka-based audit logging capabilities in Ranger, which is essential for comprehensive audit logging.