[HIVE-9234] HiveServer2 leaks FileSystem objects in FileSystem.CACHE

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.14.0
• Fix Version/s: 1.0.0
• Component/s: HiveServer2
• Assignee: Vaibhav Gumashta
• Reporter: Vaibhav Gumashta
• Created: 30/Dec/14 23:09
• Updated: 19/Feb/15 18:21
• Resolved: 15/Jan/15 19:04

Description

Running over extended period (48+ hrs), we've noticed HiveServer2 leaking FileSystem objects in FileSystem.CACHE. Linked jiras were previous attempts to fix it, but the issue still seems to be there. A workaround is to disable the caching (by setting fs.hdfs.impl.disable.cache and fs.file.impl.disable.cache to true), but creating new FileSystem objects is expensive.

Issue Links

• relates to: HIVE-4501 HS2 memory leak - FileSystem objects in FileSystem.CACHE
• relates to: HIVE-6312 doAs with plain sasl auth should be session aware
• relates to: HIVE-3098 Memory leak from large number of FileSystem instances in FileSystem.CACHE
• links to: Review Board