[KAFKA-7458] Avoid enforced processing during bootstrap phase

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: streams
• Assignee: John Roesler
• Reporter: Guozhang Wang
• Created: 29/Sep/18 00:50
• Updated: 07/Jul/21 22:53
• Resolved: 07/Jul/21 22:53

Description

In KAFKA-3514, we introduced a new config for allowing users to delay enforcing processing without all input topic partitions to have data. This config's default value is 0, which means that as long as the first fetch does not contains some records for all the partitions it will fall into enforced processing immediately, which is a high risk especially under bootstrap case.

We should consider leveraging on pause / resume to make sure that upon starting, some partition indeed does not have any data before we fall into enforced processing

Issue Links

• is related to: KAFKA-3514 Stream timestamp computation needs some further thoughts
• is related to: KAFKA-4113 Allow KTable bootstrap
• is related to: KAFKA-8478 Poll for more records before forced processing
• relates to: KAFKA-10091 Improve task idling