[SPARK-40186] mergedShuffleCleaner should have been shutdown before db closed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Component/s: Spark Core
• Target Version/s: 3.4.0
• Assignee: Yang Jie
• Reporter: Yang Jie
• Created: 23/Aug/22 03:27
• Updated: 08/Sep/22 01:58
• Resolved: 08/Sep/22 01:58

Description

Should ensure `RemoteBlockPushResolver#mergedShuffleCleaner` have been shutdown before `RemoteBlockPushResolver#db` closed, otherwise, `RemoteBlockPushResolver#applicationRemoved` may perform delete operations on a closed db.

 

https://github.com/apache/spark/pull/37610#discussion_r951185256

 

Issue Links

• links to: [Github] Pull Request #37624 (LuciferYang)