Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3382

Memory leak when files are not cleanly closed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.15.0
    • 0.17.0
    • None
    • None
    • Reviewed
    • Fixed a memory leak associated with 'abandoned' files (i.e. not cleanly closed). This held up significant amounts of memory depending on activity and how long NameNode has been running.

    Description

      FSNamesystem.internalReleaseCreate() in invoked on files that are open for writing but not cleanly closed. e.g. when client invokes abandonFileInProgress() or when lease expires. It deletes the last block if it has a length of zero. The block is deleted from the file INode but not from blocksMap. Then leaves a reference to such file until NameNode is restarted. When this happens HADOOP-3381 multiplies amount of memory leak.

      Attachments

        1. memleak.txt
          2 kB
          Raghu Angadi
        2. HADOOP-3382.patch
          0.7 kB
          Raghu Angadi
        3. HADOOP-3382.patch
          0.7 kB
          Raghu Angadi
        4. HADOOP-3382.patch
          1 kB
          Raghu Angadi

        Activity

          People

            rangadi Raghu Angadi
            rangadi Raghu Angadi
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: