Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6528

Memory leak for HistoryFileManager.getJobSummary()

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      We meet memory leak issues for JHS in a large cluster which is caused by code below doesn't release FSDataInputStream in exception case. MAPREDUCE-6273 should fix most cases that exceptions get thrown. However, we still need to fix the memory leak for occasional case.

       
      private String getJobSummary(FileContext fc, Path path) throws IOException {
          Path qPath = fc.makeQualified(path);
          FSDataInputStream in = fc.open(qPath);
          String jobSummaryString = in.readUTF();
          in.close();
          return jobSummaryString;
        }
      

      Attachments

        1. MAPREDUCE-6528.patch
          1 kB
          Junping Du

        Activity

          People

            junping_du Junping Du
            junping_du Junping Du
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: