[KAFKA-7566] Add sidecar job to leader (or a random single follower) only

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Component/s: streams
• Assignee: Unassigned
• Reporter: Boyang Chen
• Created: 29/Oct/18 19:22
• Updated: 15/May/20 01:50
• Resolved: 15/May/20 01:50

Description

Hey there,

recently we need to add an archive job to a streaming application. The caveat is that we need to make sure only one instance is doing this task to avoid potential race condition, and we also don't want to schedule it as a regular stream task so that we will be blocking normal streaming operation. 

Although we could do so by doing a zk lease, I'm raising the case here since this could be some potential use case for streaming job also. For example, there are some `leader specific` operation we could schedule in DSL instead of adhoc manner.

Let me know if you think this makes sense to you, thank you!