Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12984

BlockPoolSlice can leak in a mini dfs cluster

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.5
    • 3.1.0
    • None
    • None
    • Reviewed

    Description

      When running some unit tests for storm we found that we would occasionally get out of memory errors on the HDFS integration tests.

      When I got a heap dump I found that the ShutdownHookManager was full of BlockPoolSlice$1 instances. Which hold a reference to the BlockPoolSlice which then in turn holds a reference to the DataNode etc....

      It looks like when shutdown is called on the BlockPoolSlice there is no way to remove the shut down hook in because no reference to it is saved.

      Attachments

        1. Screen Shot 2018-01-05 at 5.31.52 PM.png
          422 kB
          Ajay Kumar
        2. Screen Shot 2018-01-05 at 5.26.54 PM.png
          277 kB
          Ajay Kumar
        3. Screen Shot 2018-01-05 at 4.38.06 PM.png
          82 kB
          Ajay Kumar
        4. HDFS-12984.001.patch
          2 kB
          Ajay Kumar

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ajayydv Ajay Kumar
            revans2 Robert Joseph Evans
            Votes:
            0 Vote for this issue
            Watchers:
            8 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment