[SPARK-34592] Mark indeterminate RDD in Web UI

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: Web UI
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 02/Mar/21 08:38
• Updated: 05/May/22 23:47
• Resolved: 05/Mar/21 17:32

Description

It is somehow hard to track which part is indeterminate in a graph of RDDs. In some cases we may need to track indeterminate RDDs. For example, indeterminate map stage fails and Spark is unable to fallback some parent stages. If Web UI can show up indeterminate RDD like cached RDD, it could be useful to track it.

Issue Links

• causes: SPARK-39100 RDD DeterministicLevel Needs Backward Compatibility
• links to: [Github] Pull Request #31707 (viirya)
• links to: [Github] Pull Request #31707 (viirya)