[SPARK-47363] Initial State without state reader implementation for State API v2.

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Structured Streaming
• Labels: pull-request-available
• Epic Link: Structured Streaming - Arbitrary State API v2
• Assignee: Jing Zhan
• Reporter: Jing Zhan
• Created: 12/Mar/24 17:13
• Updated: 28/Mar/24 05:51
• Resolved: 28/Mar/24 05:51

Description

This PR adds support for users to provide a Dataframe that can be used to instantiate state for the query in the first batch for arbitrary state API v2.

Note that populating the initial state will only happen for the first batch of the new streaming query. Trying to re-initialize state for the same grouping key will result in an error.

Issue Links

• links to: GitHub Pull Request #45467