[SPARK-45747] Support session window aggregation in state reader

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 4.0.0
• Component/s: Structured Streaming
• Labels: pull-request-available
• Assignee: Chaoqin Li
• Reporter: Chaoqin Li
• Created: 01/Nov/23 02:44
• Updated: 16/Nov/23 06:29
• Resolved: 16/Nov/23 06:29

Description

We are introducing state reader in SPARK-45511, but currently session window operator is not supported because the numColPrefixKey is unknown. We can read the operator state metadata introduced in SPARK-45558 to determine the number of prefix columns and load the state of session window correctly.

Issue Links

• links to: GitHub Pull Request #43788