[HADOOP-15565] ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.1.4, 3.2.2, 2.10.1
• Assignee: Jinglun Assign to me
• Reporter: Jinglun
• Created: 27/Jun/18 02:50
• Updated: 13/May/20 04:43
• Resolved: 13/May/20 04:43

Description

ViewFileSystem.close() does nothing but remove itself from FileSystem.CACHE. It's children filesystems are cached in FileSystem.CACHE and shared by all the ViewFileSystem instances. We could't simply close all the children filesystems because it will break the semantic of FileSystem.newInstance().
We might add an inner cache to ViewFileSystem, let it cache all the children filesystems. The children filesystems are not shared any more. When ViewFileSystem is closed we close all the children filesystems in the inner cache. The ViewFileSystem is still cached by FileSystem.CACHE so there won't be too many FileSystem instances.

The FileSystem.CACHE caches the ViewFileSysem instance and the other instances(the children filesystems) are cached in the inner cache.

Issue Links

• duplicates: HDFS-14645 ViewFileSystem should close the child FileSystems in close()
• is duplicated by: HDFS-10323 transient deleteOnExit failure in ViewFileSystem due to close() ordering