[HDFS-10549] Correctly revoke file leases when closing files

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.1
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: hdfs-client
• Hadoop Flags: Reviewed
• Assignee: Yiqun Lin
• Reporter: Yiqun Lin
• Created: 20/Jun/16 02:30
• Updated: 30/Aug/16 01:14
• Resolved: 17/Aug/16 23:04

Description

As HADOOP-13264 memtioned, the code dfsClient.endFileLease(fileId) in DFSOutputStream will not be executed when the IOException happened in closeImpl().

  public void close() throws IOException {
    synchronized (this) {
      try (TraceScope ignored =
          dfsClient.newPathTraceScope("DFSOutputStream#close", src)) {
        closeImpl();
      }
    }
    dfsClient.endFileLease(fileId);
    }
  }

This will cause that the files not be closed in DFSClient and finally lead to the memory leak. In DFSStripedOutputStream, it existed the same problem.

Issue Links

• is duplicated by: HADOOP-13264 Hadoop HDFS - DFSOutputStream close method fails to clean up resources in case no hdfs datanodes are accessible
• is related to: HDFS-6532 Intermittent test failure org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt