[DRILL-5977] predicate pushdown support kafkaMsgOffset

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: Storage - Kafka
• Labels: doc-complete ready-to-commit
• Assignee: Abhishek Ravi
• Reporter: AnilKumar B
• Reviewer: AnilKumar B
• Created: 20/Nov/17 01:43
• Updated: 14/Sep/18 21:54
• Resolved: 05/Jul/18 10:24

Description

As part of Kafka storage plugin review, below is the suggestion from Paul.

Does it make sense to provide a way to select a range of messages: a starting point or a count? Perhaps I want to run my query every five minutes, scanning only those messages since the previous scan. Or, I want to limit my take to, say, the next 1000 messages. Could we use a pseudo-column such as "kafkaMsgOffset" for that purpose? Maybe

SELECT * FROM <some topic> WHERE kafkaMsgOffset > 12345

Issue Links

• links to: GitHub Pull Request #1272