Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1239

New producer checklist

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.2.0
    • None
    • producer
    • None

    Description

      Here is the list of todo items we have for the new producer (in no
      particular order):
      1. Rename to org.apache.* package
      2. Discuss config approach
      3. Finalize config approach
      4. Add slf4j logging for debugging purposes
      5. Discuss metrics approach
      6. Add metrics
      7. Convert perf test to optionally use new producer
      8. Get system tests passing with new producer
      9. Write integration tests that test the producer against the real server
      10. Expand unit test coverage a bit
      11. Performance testing and analysis.
      12. Add compression support
      13. Discuss and perhaps add retry support
      14. Discuss the approach to protocol definition and perhaps refactor a bit
      15. Deeper code review
      16. Convert mirror maker

      This doesn't count general bug fixing which I assume we will do as we find
      them.

      Let's file subtasks for each of the above, so there is a single place to track what's outstanding on the new producer.

      Attachments

        1.
        		 Add mirror maker using 08 consumer and 09 producer Sub-task Closed Neha Narkhede
        2.
        		 Change producer performance tool to optionally use the new producer Sub-task Resolved Jay Kreps
        3.
        		 Add ability to existing system tests to use the new producer client Sub-task Resolved Jun Rao
        4.
        		 Add slf4j logging to new producer Sub-task Resolved Jay Kreps
        5.
        		 Add metrics to the producer Sub-task Closed Jay Kreps
        6.
        		 Integration test for the new producer Sub-task Closed Guozhang Wang
        7.
        		 Implement re-tries in new producer Sub-task Resolved Jay Kreps
        8.
        		 Implement compression in new producer Sub-task Resolved Guozhang Wang
        9.
        		 Improve error reporting in the Sender for request types Sub-task Resolved Unassigned
        10.
        		 New producer runs into out of order producer responses Sub-task Resolved Jay Kreps
        11.
        		 Producer close function no longer block until all messages be sent Sub-task Resolved Jay Kreps
        12.
        		 Integration Test for New Producer Part II: Broker Failure Handling Sub-task Closed Guozhang Wang
        13.
        		 Make metadata refresh configurable Sub-task Resolved Jay Kreps
        14.
        		 Producer halts on broker bounces Sub-task Resolved Jay Kreps
        15.
        		 Metadata Request with non-exist topic will case NPE and also inFlightRequest blocks producer shutdown Sub-task Closed Jay Kreps
        16.
        		 Force metadata update on error Sub-task Resolved Unassigned
        17.
        		 Retry Can Block Sub-task Resolved Unassigned
        18.
        		 add the new producer to existing tools Sub-task Resolved Neha Narkhede
        19.
        		 TestLogCleaning tool hangs on the new producer Sub-task Resolved Unassigned
        20.
        		 unregistered mbean exception in new producer Sub-task Resolved Jun Rao
        21.
        		 Rationalize new producer configs Sub-task Resolved Jay Kreps
        22.
        		 Add topic/broker metrics once new topic/broker is discovered Sub-task Resolved Guozhang Wang
        23.
        		 Add the compression ratio metrics in the new producer Sub-task Closed Dong Lin

        Activity

          People

            Unassigned Unassigned
            nehanarkhede Neha Narkhede
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: