[SAMZA-1312] Add Control Messages and Intermediate Stream Serde

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Xinyu Liu
• Reporter: Xinyu Liu
• Created: 26/May/17 21:24
• Updated: 28/Jun/17 18:07
• Resolved: 28/Jun/17 18:07

Description

The format of the intermediate stream message:

• IntermediateMessage => [MessageType MessageData]
  • MessageType => byte
  • MessageData => byte[]

• MessageType => [0(UserMessage), 1(Watermark), 2(EndOfStream)]
• MessageData => [UserMessage/ControlMessage]
• ControlMessage =>
  • Type => int
  • Version => int
  • Other Message Data (based on different types of control message)

For user message, we will use the user provided serde (default is the system serde). For control message, we will use JSON serde since it is built in Samza and easy to parse.

Issue Links

• is depended upon by: SAMZA-1260 Support Control Messages Across Intermediate Streams
• links to: GitHub Pull Request #207