Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12748

NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      In our production environment, the standby NN often do fullgc, through mat we found the largest object is FileSystem$Cache, which contains 7,844,890 DistributedFileSystem.
      By view hierarchy of method FileSystem.get() , I found only NamenodeWebHdfsMethods#get call FileSystem.get(). I don't know why creating different DistributedFileSystem every time instead of get a FileSystem from cache.

          case GETHOMEDIRECTORY: {
            final String js = JsonUtil.toJsonString("Path",
                FileSystem.get(conf != null ? conf : new Configuration())
                    .getHomeDirectory().toUri().getPath());
            return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
          }
      

      When we close FileSystem when GETHOMEDIRECTORY, NN don't do fullgc.

          case GETHOMEDIRECTORY: {
            FileSystem fs = null;
            try {
              fs = FileSystem.get(conf != null ? conf : new Configuration());
              final String js = JsonUtil.toJsonString("Path",
                  fs.getHomeDirectory().toUri().getPath());
              return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
            } finally {
              if (fs != null) {
                fs.close();
              }
            }
          }
      

      Attachments

        1. HDFS-12748.001.patch
          4 kB
          Weiwei Yang
        2. HDFS-12748.002.patch
          9 kB
          Weiwei Yang
        3. HDFS-12748.003.patch
          5 kB
          Weiwei Yang
        4. HDFS-12748.004.patch
          4 kB
          Weiwei Yang
        5. HDFS-12748.005.patch
          4 kB
          Weiwei Yang
        6. HDFS-12748-branch-3.1.01.patch
          4 kB
          Weiwei Yang

        Issue Links

          Activity

            People

              cheersyang Weiwei Yang
              yangjiandan Jiandan Yang
              Votes:
              2 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: