[SPARK-28760] Add end-to-end Kafka delegation token test

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Structured Streaming, Tests
• Assignee: Gabor Somogyi
• Reporter: Gabor Somogyi
• Created: 16/Aug/19 10:20
• Updated: 24/Feb/24 11:23
• Resolved: 29/Aug/19 18:52

Description

At the moment no end-to-end Kafka delegation token test exists which was mainly because of missing KDC. KDC is missing in general from the testing side so I've discovered what kind of possibilities are there. The most obvious choice is the MiniKDC inside the Hadoop library where Apache Kerby runs in the background. In this jira I would like to add Kerby to the testing area and use it to cover security related features.

Issue Links

• is related to: SPARK-47154 Fix `kafka-0-10-sql` to use `ResetSystemProperties` if `KafkaTestUtils` is used
• links to: GitHub Pull Request #25477