[HADOOP-3139] DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.16.3
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 31/Mar/08 20:09
• Updated: 08/Jul/09 16:43
• Resolved: 04/Apr/08 23:15

Description

Koji found the following:

DistributedFileSystem.close() deadlock
My dfs -ls hang.
Ctrl-Z showed a deadlock state.

"Thread-0":
        at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190)
        - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
        at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231)
        - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
        at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169)
        at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154)
        - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer)
"main":
        at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201)
        - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
        at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085)
        at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192)
        - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
        at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712)

Found 1 deadlock.

FileSystem.closeAll() warning

08/03/31 16:48:42 INFO fs.FileSystem: FileSystem.closeAll() threw an exception:
java.io.IOException: HftpFileSystem(=org.apache.hadoop.dfs.HftpFileSystem@111111) and
Key(=null@hftp://namenode-nn:4444) do not match.