[KAFKA-8665] WorkerSourceTask race condition when rebalance occurs before task has started

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: connect
• Assignee: Unassigned
• Reporter: Chris Cranford
• Created: 15/Jul/19 16:41
• Updated: 15/Jul/19 16:41

Description

In our project we have several SourceTask implementations that perform a set of sequential steps in the SourceTask#start call.  It's possible that during these sequential operations, a rebalance occurs which leads to the situation where the WorkerSourceTask transitions to the state where startedShutdownBeforeStartCompleted=true and then the runtime starts a brand new task.

This is mostly a problem around specific named resources that are registered when a call to SourceTask#start happens and those same resources are unregistered later when the call to SourceTask#stop occurs. 

For us specifically, this is a problem with JMX resource registration/unregistration.  We register those beans at the end of the call to SourceTask#start and unregister in the call to stop.  Due to the order of start/stop pairs combined with where a rebalance is triggered, this leads to 

1. Register JMX beans when SourceTask A1 is started.
2. Register JMX beans when SourceTask A2 is started with rebalance.
   1. JMX beans failed to register as they're already registered.
3. SourceTask A1 finally stops, triggers unregistering JMX beans

 

In our use case we're experiencing a problem with the registration/unregistration of JMX resources with the nature of how a rebalance is triggered while the task hasn't yet fully started and never gets stopped