Details
-
Sub-task
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In SAMZA-552, window operator is designed to have a window metadata store internally to keep all windows' metadata in retention. This metadata store should be a KV store w/ a changelog (i.e. LoggedStore), with the following definition of key and value:
- Time-based window metadata store:
- Key: window start timestamp
- Value: {start_time, start_offset, end_time, last_offset}
- Tuple-based window metadata store:
- Key: window start offset
- Value: {start_offset, last_offset}