[KAFKA-6150] Make Repartition Topics Transient

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: streams
• Labels: operability
• Assignee: Guozhang Wang
• Reporter: Guozhang Wang
• Created: 31/Oct/17 00:54
• Updated: 07/Jul/18 00:04
• Resolved: 04/Dec/17 18:21

Description

Unlike changelog topics, the repartition topics could just be short-lived. Today users have different ways to configure them with short retention such as enforce a short retention period or use AppendTime for repartition topics. All these would be cumbersome and Streams should just do this for the users.

One way to do it is use the “purgeData” admin API (KIP-107) such that after the offset of the input topics are committed, if the input topics are actually repartition topics, we would purge the data immediately.

Issue Links

• depends upon: KAFKA-5925 Adding records deletion operation to the new Admin Client API
• is duplicated by: KAFKA-6038 Repartition topics could be much more transient
• links to: GitHub Pull Request #4270
• links to: GitHub Pull Request #4315
• mentioned in: Page Loading...