Thanks for the review. Attaching patch v2.
- currentOffset actually can be none. A fetcher can be removed after the multi-fetch request is made.
– addFetcher actually always adds a fetcher, but not always creates a new fetcher thread. I see the naming is a bit confusing. Renamed AbstractFetcher to AbstractFetcherThread.
– Fetchermanager is maintaining 1 or more fetcherThreads per source broker. Be default, there is 1 fetcherThread per broker. However, for higher degree of parallelism, more fetcherThreads can be configured. A fetcher corresponds to the fetching from 1 partition of a topic. Multiple fetchers can be added to a fetcherThread.
– We need to get the host/port from ZK for a given broker id. Such information should be cached. Will create a separate jira to address this issue.
The rest of of comments have been fixed.