Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2511 KIP-31 & KIP-32: message format change + adding timestamp to messages
  3. KAFKA-3026

KIP-32 (part 2): Changes in broker to over-write timestamp or reject message

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      The decision about this JIRA assignment is still under discussion with Jiangjie Qin. Not going to implement without his agreement.

      This JIRA includes:
      When the broker receives a message, it checks the configs:
      1. If message.timestamp.type=LogAppendTime, the server over-writes the timestamp with its current local time
      Message could be compressed or not compressed. In either case, the timestamp is always over-written to broker's current time

      2. If message.timestamp.type=CreateTime, the server calculated the difference between the current time on broker and Timestamp in the message:
      If difference is within max.message.time.difference.ms, the server will accept it and append it to the log. For compressed message, server will update the timestamp in compressed message to -1: this means that CreateTime is used and the timestamp is in each individual inner message.
      If difference is higher than max.message.time.difference.ms, the server will reject the entire batch with TimestampExceededThresholdException.

      (Actually adding the timestamp to the message and adding configs are covered by KAFKA-3025).

        Attachments

          Activity

            People

            • Assignee:
              becket_qin Jiangjie Qin
              Reporter:
              apovzner Anna Povzner
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: