[SPARK-49979] AQE hang forever when collecting twice on a failed plan

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0, 3.5.4
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Ziqi Liu
• Reporter: Ziqi Liu
• Created: 15/Oct/24 21:26
• Updated: 17/Oct/24 04:25
• Resolved: 17/Oct/24 04:25

Description

When we collect twice in a failed AQE plan, no new query stage will be created, and no stage will be submitted either. We will be waiting for a finish event forever, which will never come because that query stage has already failed in the previous run.

Issue Links

• links to: GitHub Pull Request #48484