Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2511 KIP-31 & KIP-32: message format change + adding timestamp to messages
  3. KAFKA-3025

KIP-31&KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      This JIRA is for changes for KIP-32 excluding broker checking and acting on timestamp field in a message.

      This JIRA includes:
      1. Bump up MessageAndOffset version to version 1 to:
      a. Change absolute offset to relative offset.
      b. Add time field to the message
      Timestamp => int64
      Timestamp is the number of milliseconds since Unix Epoch

      2. Add time field to both ProducerRecord and Consumer Record
      If a user specifies the timestamp in a ProducerRecord, the ProducerRecord is sent with this timestamp.
      If a user does not specify the timestamp in a ProducerRecord, the producer stamps the ProducerRecord with current time.
      ConsumerRecord will have the timestamp of the message that were stored on broker.

      3. Add two new configurations to the broker. Configuration is per topic.

      • message.timestamp.type: type of a timestamp. Possible values: CreateTime, LogAppendTime. Default: CreateTime
      • max.message.time.difference.ms: threshold for the acceptable time difference between Timestamp in the message and local time on the broker. Default: Long.MaxValue

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                becket_qin Jiangjie Qin
                Reporter:
                apovzner Anna Povzner
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: