[SPARK-48383] In KafkaOffsetReader, partition mismatch should not be an assertion

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Structured Streaming
• Labels: pull-request-available
• Assignee: Siying Dong
• Reporter: Siying Dong
• Created: 21/May/24 22:21
• Updated: 03/Jun/24 03:54
• Resolved: 03/Jun/24 03:54

Description

In KafkaOffsetReader, we assert startOffsets have the same topic partition list as assigned. However, if the user changes topic partition while the query is running, they will see the assertion. Instead, they should see an exception.

Issue Links

• links to: GitHub Pull Request #46692