[HADOOP-16665] Filesystems to be closed if they failed during initialize()

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.1, 3.1.3
• Fix Version/s: 3.3.0
• Component/s: fs, fs/s3
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 21/Oct/19 11:56
• Updated: 13/Nov/19 14:39
• Resolved: 12/Nov/19 18:19

Description

We seen this with S3A but it is likely to apply to the other file systems too:

If the file system raises an exception during initialize() then it is not close()'d. Any resources threads etc created during initialisation may be leaked.

I'm going to fix S3A to clean up internally, but the FileSystem get() and newInstance() methods should also catch and close if an IOE or RTE is raised.

Issue Links

• is depended upon by: HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
• links to: GitHub Pull Request #1671