[SPARK-43118] Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.2
• Fix Version/s: 3.5.0
• Component/s: Structured Streaming
• Assignee: Boyang Jerry Peng
• Reporter: Boyang Jerry Peng
• Created: 13/Apr/23 03:38
• Updated: 13/Apr/23 08:10
• Resolved: 13/Apr/23 08:10

Description

The assert 

 

 assert(Thread.currentThread().isInstanceOf[UninterruptibleThread]) 

 

found https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchStream.scala#L239 

 

is not needed.  The reason is the following

 

1. This assert was put there due to some issues when the old and deprecated KafkaOffsetReaderConsumer is used.  The default offset reader implementation has been changed to KafkaOffsetReaderAdmin which no longer require it run via UninterruptedThread.
2. Even if the deprecated KafkaOffsetReaderConsumer is used, there are already asserts in that impl to check if it is running via UninterruptedThread e.g. https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReaderConsumer.scala#L130 thus the assert in KafkaMicroBatchStream is redundant.

Issue Links

• links to: [Github] Pull Request #40767 (jerrypeng)
• links to: [Github] Pull Request #40767 (jerrypeng)