The Fetcher class is used internally by the KafkaConsumer to fetch records from the brokers. There is ongoing work to create a new consumer implementation with a significantly refactored threading model. The threading refactor work requires a similarly refactored Fetcher.
This task is to introduce a new class named FetchRequestManager that will be responsible for:
- Formatting fetch requests to the background thread
- Configuring the callback on fetch responses for the background thread
The response handler will collect the fetch responses from the broker and create CompletedFetch, instances, much as is done in Fetcher. The newly introduced FetchUtils will be used for both FetchRequestManager and Fetcher to keep the logic as reusable as possible.
The foreground logic will decompress the data into a Record, which will then be deserialized into a ConsumerRecord for returning to the user.