[SPARK-30835] Add support for YARN decommissioning & pre-emption

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.4.0
• Component/s: Spark Core, YARN
• Assignee: Abhishek Dixit
• Reporter: Holden Karau
• Created: 14/Feb/20 20:31
• Updated: 16/Jun/22 17:45
• Resolved: 14/Jun/22 02:52

Issue Links

• causes: SPARK-39491 Hadoop 2.7 build fails due to org.apache.hadoop.yarn.api.records.NodeState.DECOMMISSIONING
• incorporates: SPARK-39024 Add support for YARN decommissioning when ESS is Enabled
• incorporates: SPARK-39018 Add support for YARN decommissioning when ESS is Disabled
• is related to: SPARK-30873 Handling Node Decommissioning for Yarn cluster manager in Spark
• links to: [Github] Pull Request #35683 (abhishekd0907)
• links to: [Github] Pull Request #35683 (abhishekd0907)
• links to: [Github] Pull Request #36890 (abhishekd0907)