[KAFKA-3025] KIP-31&KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.0.0
• Assignee: Jiangjie Qin
• Reporter: Anna Povzner
• Created: 21/Dec/15 22:14
• Updated: 25/Feb/16 20:15
• Resolved: 19/Feb/16 15:57

Description

This JIRA is for changes for KIP-32 excluding broker checking and acting on timestamp field in a message.

This JIRA includes:
1. Bump up MessageAndOffset version to version 1 to:
a. Change absolute offset to relative offset.
b. Add time field to the message
Timestamp => int64
Timestamp is the number of milliseconds since Unix Epoch

2. Add time field to both ProducerRecord and Consumer Record
If a user specifies the timestamp in a ProducerRecord, the ProducerRecord is sent with this timestamp.
If a user does not specify the timestamp in a ProducerRecord, the producer stamps the ProducerRecord with current time.
ConsumerRecord will have the timestamp of the message that were stored on broker.

3. Add two new configurations to the broker. Configuration is per topic.

• message.timestamp.type: type of a timestamp. Possible values: CreateTime, LogAppendTime. Default: CreateTime
• max.message.time.difference.ms: threshold for the acceptable time difference between Timestamp in the message and local time on the broker. Default: Long.MaxValue

Issue Links

• is related to: KAFKA-3259 KIP-31/KIP-32 clean-ups