Hadoop Common
  1. Hadoop Common
  2. HADOOP-3998

Got an exception from ClientFinalizer when the JT is terminated

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This happens when we terminate the JT using control-C. It throws the following exception

      Exception closing file my-file
      java.io.IOException: Filesystem closed
              at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:193)
              at org.apache.hadoop.hdfs.DFSClient.access$700(DFSClient.java:64)
              at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2868)
              at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2837)
              at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:808)
              at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:205)
              at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:253)
              at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1367)
              at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:234)
              at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:219)
      

      Note that my-file is some file used by the JT.

      Also if there is some file renaming done, then the exception states that the earlier file does not exist. I am not sure if this is a MR issue or a DFS issue. Opening this issue for investigation.

      1. closeAll3.patch
        5 kB
        dhruba borthakur
      2. closeAll2.patch
        15 kB
        dhruba borthakur
      3. closeAll.patch
        0.5 kB
        dhruba borthakur
      4. closeAll.patch
        5 kB
        dhruba borthakur
      5. closeAll.patch
        12 kB
        dhruba borthakur

        Issue Links

          Activity

          Amar Kamat created issue -
          Stefan Will made changes -
          Field Original Value New Value
          Affects Version/s 0.19.0 [ 12313211 ]
          Stefan Will made changes -
          Component/s dfs [ 12310710 ]
          dhruba borthakur made changes -
          Fix Version/s 0.19.2 [ 12313650 ]
          dhruba borthakur made changes -
          Assignee dhruba borthakur [ dhruba ]
          dhruba borthakur made changes -
          Attachment closeAll.patch [ 12400399 ]
          dhruba borthakur made changes -
          Attachment closeAll.patch [ 12400810 ]
          dhruba borthakur made changes -
          Link This issue relates to HADOOP-5311 [ HADOOP-5311 ]
          Robert Chansler made changes -
          Fix Version/s 0.20.0 [ 12313438 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          Fix Version/s 0.18.4 [ 12313628 ]
          dhruba borthakur made changes -
          Attachment closeAll.patch [ 12400960 ]
          dhruba borthakur made changes -
          Attachment closeAll2.patch [ 12401381 ]
          dhruba borthakur made changes -
          Attachment closeAll3.patch [ 12401658 ]
          dhruba borthakur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hairong Kuang made changes -
          Hadoop Flags [Reviewed]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Fix Version/s 0.18.4 [ 12313628 ]
          Resolution Fixed [ 1 ]
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development