Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10549

Correctly revoke file leases when closing files

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As HADOOP-13264 memtioned, the code dfsClient.endFileLease(fileId) in DFSOutputStream will not be executed when the IOException happened in closeImpl().

        public void close() throws IOException {
          synchronized (this) {
            try (TraceScope ignored =
                dfsClient.newPathTraceScope("DFSOutputStream#close", src)) {
              closeImpl();
            }
          }
          dfsClient.endFileLease(fileId);
          }
        }
      

      This will cause that the files not be closed in DFSClient and finally lead to the memory leak. In DFSStripedOutputStream, it existed the same problem.

        Attachments

        1. HDFS-10549.005.patch
          7 kB
          Yiqun Lin
        2. HDFS-10549.004.patch
          6 kB
          Yiqun Lin
        3. HDFS-10549.003.patch
          3 kB
          Yiqun Lin
        4. HDFS-10549.002.patch
          0.9 kB
          Yiqun Lin
        5. HDFS-10549.001.patch
          3 kB
          Yiqun Lin

          Issue Links

            Activity

              People

              • Assignee:
                linyiqun Yiqun Lin
                Reporter:
                linyiqun Yiqun Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: