We are refactoring the current KafkaConsumer and making it more asynchronous. This is the master Jira to track the project's progress; subtasks will be linked to this ticket. Please review the design document and feel free to use this thread for discussion.
The design document is here: https://cwiki.apache.org/confluence/display/KAFKA/Proposal%3A+Consumer+Threading+Model+Refactor
The original email thread is here: https://lists.apache.org/thread/13jvwzkzmb8c6t7drs4oj2kgkjzcn52l
I will continue to update the 1pager as reviews and comments come.