Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1326 New consumer checklist
  3. KAFKA-2123

Make new consumer offset commit API use callback + future

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.0
    • Component/s: clients, consumer
    • Labels:
      None

      Description

      The current version of the offset commit API in the new consumer is

      void commit(offsets, commit type)

      where the commit type is either sync or async. This means you need to use sync if you ever want confirmation that the commit succeeded. Some applications will want to use asynchronous offset commit, but be able to tell when the commit completes.

      This is basically the same problem that had to be fixed going from old consumer -> new consumer and I'd suggest the same fix using a callback + future combination. The new API would be

      Future<Void> commit(Map<TopicPartition, Long> offsets, ConsumerCommitCallback callback);

      where ConsumerCommitCallback contains a single method:

      public void onCompletion(Exception exception);

      We can provide shorthand variants of commit() for eliding the different arguments.

        Attachments

        1. KAFKA-2123.patch
          47 kB
          Ewen Cheslack-Postava
        2. KAFKA-2123.patch
          207 kB
          Jason Gustafson
        3. KAFKA-2123_2015-07-14_18:21:38.patch
          317 kB
          Jason Gustafson
        4. KAFKA-2123_2015-07-14_13:20:25.patch
          293 kB
          Jason Gustafson
        5. KAFKA-2123_2015-07-13_18:45:08.patch
          291 kB
          Jason Gustafson
        6. KAFKA-2123_2015-07-11_17:33:59.patch
          274 kB
          Jason Gustafson
        7. KAFKA-2123_2015-05-29_11:11:05.patch
          102 kB
          Ewen Cheslack-Postava
        8. KAFKA-2123_2015-05-04_22:51:48.patch
          102 kB
          Ewen Cheslack-Postava
        9. KAFKA-2123_2015-05-04_09:39:50.patch
          81 kB
          Ewen Cheslack-Postava
        10. KAFKA-2123_2015-05-01_19:33:19.patch
          76 kB
          Ewen Cheslack-Postava
        11. KAFKA-2123_2015-04-30_11:23:05.patch
          57 kB
          Ewen Cheslack-Postava

          Activity

            People

            • Assignee:
              hachikuji Jason Gustafson
              Reporter:
              ewencp Ewen Cheslack-Postava
              Reviewer:
              Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: