Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10422

Provide a `timesForOffsets` operation in kafka consumer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • clients, core
    • None

    Description

      The kafka consumer already provides an operation to quickly lookup the offsets by timestamp by using the `offsetsForTimes` operation.

      However there are use cases where the inverse operation would be useful: having a set of offsets, I would like to lookup the ingestion timestamps for all these messages. Currently it would require fetching all these message by random access to retrieve the timestamps.

      I propose to add the `timesForOffsets` operation to the kafka consumer. The operation signature would be equivalent to `offsetsForTimes`: given a mapping from partition to the offset to look up, it would return a mapping from partition to the timestamp and offset of the message at the requested offset. null would be returned for the partition if there is no message at this offset.

      I think it would require an API change in `ListOffset`, so maybe it does require a KIP?

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              guillaumebort Guillaume Bort
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: