[SPARK-20792] Support same timeout operations in mapGroupsWithState function in batch queries as in streaming queries

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.0
• Component/s: Structured Streaming
• Target Version/s: 2.2.0
• Assignee: Tathagata Das
• Reporter: Tathagata Das
• Created: 18/May/17 08:23
• Updated: 21/May/17 23:07
• Resolved: 21/May/17 23:07

Description

Currently, in the batch queries, timeout is disabled (i.e. GroupStateTimeout.NoTimeout) which means any GroupState.setTimeout*** operation would throw UnsupportedOperationException. This makes it weird when converting a streaming query into a batch query by changing the input DF from streaming to a batch DF. If the timeout was enabled and used, then the batch query will start throwing UnsupportedOperationException.

Issue Links

• links to: [Github] Pull Request #18024 (tdas)