[SPARK-44252] Add error class for the case when loading state from DFS fails

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core, SQL
• Assignee: Lucy Yao
• Reporter: Lucy Yao
• Created: 29/Jun/23 20:14
• Updated: 21/Jul/23 00:22
• Resolved: 21/Jul/23 00:22

Description

This is part of https://github.com/apache/spark/pull/41705.

Wrap the exception during the loading state, to assign error class properly. With assigning error class, we can classify the errors which help us to determine what errors customers are struggling much. StateStoreProvider.getStore() & StateStoreProvider.getReadStore() is the entry point.

This ticket also covers failedToReadDeltaFileError and failedToReadSnapshotFileError from https://issues.apache.org/jira/browse/SPARK-36305.

Issue Links

• links to: [Github] Pull Request #41705 (lucyyao-db)