Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10616 Improve performance of path handling
  3. HDFS-10673

Optimize FSPermissionChecker's internal path usage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 2.7.4, 3.0.0-alpha1
    • hdfs
    • None
    • Reviewed

    Description

      The INodeAttributeProvider and AccessControlEnforcer features degrade performance and generate excessive garbage even when neither is used. Main issues:

      1. A byte[][] of components is unnecessarily created. Each path component lookup converts a subrange of the byte[][] to a new String[] - then not used by default attribute provider.
      2. Subaccess checks are insanely expensive. The full path of every subdir is created by walking up the inode tree, creating a INode[], building a string by converting each inode's byte[] name to a string, etc. Which will only be used if there's an exception.

      The expensive of #1 should only be incurred when using the provider/enforcer feature. For #2, paths should be created on-demand for exceptions.

      Attachments

        1. HDFS-10673-branch-2.7.00.patch
          21 kB
          Zhe Zhang
        2. HDFS-10673.patch
          18 kB
          Daryn Sharp
        3. HDFS-10673.2.patch
          20 kB
          Daryn Sharp
        4. HDFS-10673.1.patch
          21 kB
          Daryn Sharp

        Issue Links

          Activity

            People

              daryn Daryn Sharp
              daryn Daryn Sharp
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: