Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7454

Reduce memory footprint for AclEntries in NameNode

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-5620 indicated a GlobalAclSet containing unique AclFeature can be de-duplicated to save the memory in NameNode. However it was not implemented at that time.

      This Jira re-proposes same implementation, along with de-duplication of unique AclEntry across all ACLs.

      One simple usecase is:
      A mapreduce user's home directory with the set of default ACLs, under which lot of other files/directories could be created when jobs is run. Here all the default ACLs of parent directory will be duplicated till the explicit delete of those ACLs. With de-duplication,only one object will be in memory for the same Entry across all ACLs of all files/directories.

        Attachments

        1. HDFS-7454-004.patch
          22 kB
          Vinayakumar B
        2. HDFS-7454-003.patch
          15 kB
          Vinayakumar B
        3. HDFS-7454-002.patch
          13 kB
          Vinayakumar B
        4. HDFS-7454-001.patch
          6 kB
          Vinayakumar B

          Issue Links

            Activity

              People

              • Assignee:
                vinayakumarb Vinayakumar B
                Reporter:
                vinayakumarb Vinayakumar B
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: