Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1326 New consumer checklist
  3. KAFKA-2123

Make new consumer offset commit API use callback + future

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 0.9.0.0
    • clients, consumer
    • None

    Description

      The current version of the offset commit API in the new consumer is

      void commit(offsets, commit type)

      where the commit type is either sync or async. This means you need to use sync if you ever want confirmation that the commit succeeded. Some applications will want to use asynchronous offset commit, but be able to tell when the commit completes.

      This is basically the same problem that had to be fixed going from old consumer -> new consumer and I'd suggest the same fix using a callback + future combination. The new API would be

      Future<Void> commit(Map<TopicPartition, Long> offsets, ConsumerCommitCallback callback);

      where ConsumerCommitCallback contains a single method:

      public void onCompletion(Exception exception);

      We can provide shorthand variants of commit() for eliding the different arguments.

      Attachments

        1. KAFKA-2123.patch
          47 kB
          Ewen Cheslack-Postava
        2. KAFKA-2123_2015-04-30_11:23:05.patch
          57 kB
          Ewen Cheslack-Postava
        3. KAFKA-2123_2015-05-01_19:33:19.patch
          76 kB
          Ewen Cheslack-Postava
        4. KAFKA-2123_2015-05-04_09:39:50.patch
          81 kB
          Ewen Cheslack-Postava
        5. KAFKA-2123_2015-05-04_22:51:48.patch
          102 kB
          Ewen Cheslack-Postava
        6. KAFKA-2123_2015-05-29_11:11:05.patch
          102 kB
          Ewen Cheslack-Postava
        7. KAFKA-2123.patch
          207 kB
          Jason Gustafson
        8. KAFKA-2123_2015-07-11_17:33:59.patch
          274 kB
          Jason Gustafson
        9. KAFKA-2123_2015-07-13_18:45:08.patch
          291 kB
          Jason Gustafson
        10. KAFKA-2123_2015-07-14_13:20:25.patch
          293 kB
          Jason Gustafson
        11. KAFKA-2123_2015-07-14_18:21:38.patch
          317 kB
          Jason Gustafson

        Activity

          People

            hachikuji Jason Gustafson
            ewencp Ewen Cheslack-Postava
            Guozhang Wang Guozhang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: