[SPARK-24667] If folders managed by DiskBlockManager are deleted manually, shell throws FileNotFoundException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.3.1
• Component/s: Block Manager, Spark Core
• Assignee: Unassigned
• Reporter: wuyonghua
• Created: 27/Jun/18 09:20
• Updated: 12/Dec/22 18:10
• Resolved: 11/Jul/18 06:33

Description

1. In spark shell
2. Run a join query, which writes some data to folders managed by DiskBlockManager.
3. Delete some of the folders, e.g.tmp\blockmgr-7145fe5a-757c-4640-a032-047c7f1d4080\2d
4. run the same query again, it throws FileNotFoundException
5. Suggest to check the folder existence when found a folder in subDirs