Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3139

DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.16.3
    • None
    • None
    • Reviewed

    Description

      Koji found the following:

      DistributedFileSystem.close() deadlock
      My dfs -ls hang.
      Ctrl-Z showed a deadlock state.

      "Thread-0":
              at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190)
              - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
              at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231)
              - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
              at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169)
              at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154)
              - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer)
      "main":
              at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201)
              - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
              at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085)
              at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192)
              - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
              at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698)
              at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712)
      
      Found 1 deadlock.
      

      FileSystem.closeAll() warning

      08/03/31 16:48:42 INFO fs.FileSystem: FileSystem.closeAll() threw an exception:
      java.io.IOException: HftpFileSystem(=org.apache.hadoop.dfs.HftpFileSystem@111111) and
      Key(=null@hftp://namenode-nn:4444) do not match.
      

      Attachments

        1. 3139_20080331.patch
          1 kB
          Tsz-wo Sze
        2. 3139_20080402b.patch
          3 kB
          Tsz-wo Sze
        3. 3139_20080403.patch
          3 kB
          Tsz-wo Sze
        4. 3139_20080403_0.16.patch
          3 kB
          Tsz-wo Sze

        Activity

          People

            szetszwo Tsz-wo Sze
            szetszwo Tsz-wo Sze
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: