Hadoop Common
  1. Hadoop Common
  2. HADOOP-3998

Got an exception from ClientFinalizer when the JT is terminated

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This happens when we terminate the JT using control-C. It throws the following exception

      Exception closing file my-file
      java.io.IOException: Filesystem closed
              at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:193)
              at org.apache.hadoop.hdfs.DFSClient.access$700(DFSClient.java:64)
              at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2868)
              at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2837)
              at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:808)
              at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:205)
              at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:253)
              at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1367)
              at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:234)
              at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:219)
      

      Note that my-file is some file used by the JT.

      Also if there is some file renaming done, then the exception states that the earlier file does not exist. I am not sure if this is a MR issue or a DFS issue. Opening this issue for investigation.

      1. closeAll3.patch
        5 kB
        dhruba borthakur
      2. closeAll2.patch
        15 kB
        dhruba borthakur
      3. closeAll.patch
        0.5 kB
        dhruba borthakur
      4. closeAll.patch
        5 kB
        dhruba borthakur
      5. closeAll.patch
        12 kB
        dhruba borthakur

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development