Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3545

DFSClient leak due to malfunctioning of FileSystem Cache

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha, 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      For every FileSystem.get new FileSystem object is getting created even though the UGI object passed has same name. This is creating the lot of FileSystem objects and cached in FileSystem cache instead of using the same cached object .

      This is causing the Cache to grow in size causing OOME

      This behaviour can be seen in Mapred and Hive components also since they use FileSystem.get in the described fashion

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                amithdk amith
              • Votes:
                0 Vote for this issue
                Watchers:
                29 Start watching this issue

                Dates

                • Created:
                  Updated: