Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2511 KIP-31 & KIP-32: message format change + adding timestamp to messages
  3. KAFKA-3025

KIP-31&KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord


    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s:
    • Component/s: None
    • Labels:


      This JIRA is for changes for KIP-32 excluding broker checking and acting on timestamp field in a message.

      This JIRA includes:
      1. Bump up MessageAndOffset version to version 1 to:
      a. Change absolute offset to relative offset.
      b. Add time field to the message
      Timestamp => int64
      Timestamp is the number of milliseconds since Unix Epoch

      2. Add time field to both ProducerRecord and Consumer Record
      If a user specifies the timestamp in a ProducerRecord, the ProducerRecord is sent with this timestamp.
      If a user does not specify the timestamp in a ProducerRecord, the producer stamps the ProducerRecord with current time.
      ConsumerRecord will have the timestamp of the message that were stored on broker.

      3. Add two new configurations to the broker. Configuration is per topic.

      • message.timestamp.type: type of a timestamp. Possible values: CreateTime, LogAppendTime. Default: CreateTime
      • max.message.time.difference.ms: threshold for the acceptable time difference between Timestamp in the message and local time on the broker. Default: Long.MaxValue


          Issue Links



              • Assignee:
                becket_qin Jiangjie Qin
                apovzner Anna Povzner
              • Votes:
                0 Vote for this issue
                8 Start watching this issue


                • Created: