[SPARK-21370] Avoid doing anything on HDFSBackedStateStore.abort() when there are no updates to commit

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.1
• Fix Version/s: 2.3.0
• Component/s: Structured Streaming
• Target Version/s: 2.3.0
• Assignee: Burak Yavuz
• Reporter: Burak Yavuz
• Created: 11/Jul/17 00:41
• Updated: 12/Jul/17 07:39
• Resolved: 12/Jul/17 07:39

Description

During Streaming Aggregation, we have two StateStores per task, one used as read-only in StateStoreRestoreExec, and one read-write used in `StateStoreSaveExec`. `StateStore.abort` will be called for these StateStores if they haven't committed their results. We need to make sure that abort in read-only store after a commit in the read-write store doesn't
accidentally lead to the deletion of state.

This JIRA proposes a test for this case. StateStore implementations should successfully handle this use case.

Issue Links

• links to: [Github] Pull Request #18595 (brkyvz)
• links to: [Github] Pull Request #18603 (brkyvz)