• Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:


      We should enable consumers to save their position themselves. This would be useful for consumers that need to store consumed data so they can store the data and the position together, this gives a poor man's "transactionality" since any data loss on the consumer will also rewind the position to the previous position so the two are always in sync.

      Two ways to do this:
      1. Add an OffsetStorage interface and have the zk storage implement this. The user can override this by providing an OffsetStorage implementation of their own to change how values are stored.
      2. Make commit() return the position offset vector and add a setPosition(List<Long>) method to initialize the position.

      Let's figure out any potential problems with this, and work out the best approach.


        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        989d 3h 8m 1 Jay Kreps 17/Jul/14 23:35
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13052380 ] no-reopen-closed, patch-avail [ 13055109 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12640450 ] Apache Kafka Workflow [ 13052380 ]
        Jay Kreps made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Jay Kreps added a comment -

        This is being done in the new consumer.

        Jay Kreps added a comment - This is being done in the new consumer.
        Neha Narkhede made changes -
        Field Original Value New Value
        Parent KAFKA-364 [ 12560271 ]
        Issue Type New Feature [ 2 ] Sub-task [ 7 ]
        Jay Kreps created issue -


          • Assignee:
            Jay Kreps
            Jay Kreps
          • Votes:
            0 Vote for this issue
            2 Start watching this issue


            • Created: