[KAFKA-1893] Allow regex subscriptions in the new consumer

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.9.0.0
• Component/s: consumer
• Assignee: Ashish Singh
• Reporter: Jay Kreps
• Reviewer: Gwen Shapira
• Created: 23/Jan/15 18:23
• Updated: 10/Sep/15 20:11
• Resolved: 10/Sep/15 20:11

Description

The consumer needs to handle subscribing to regular expressions. Presumably this would be done as a new api,

  void subscribe(java.util.regex.Pattern pattern);

Some questions/thoughts to work out:

• It should not be possible to mix pattern subscription with partition subscription.
• Is it allowable to mix this with normal topic subscriptions? Logically this is okay but a bit complex to implement.
• We need to ensure we regularly update the metadata and recheck our regexes against the metadata to update subscriptions for new topics that are created or old topics that are deleted.

Issue Links

• is blocked by: KAFKA-2275 Add a ListTopics() API to the new consumer
• is blocked by: KAFKA-2381 Possible ConcurrentModificationException while unsubscribing from a topic in new consumer
• is blocked by: KAFKA-2413 New consumer's subscribe(Topic...) api fails if called more than once