Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10616 Improve performance of path handling
  3. HDFS-10673

Optimize FSPermissionChecker's internal path usage

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 2.7.4, 3.0.0-alpha1
    • hdfs
    • None
    • Reviewed

    Description

      The INodeAttributeProvider and AccessControlEnforcer features degrade performance and generate excessive garbage even when neither is used. Main issues:

      1. A byte[][] of components is unnecessarily created. Each path component lookup converts a subrange of the byte[][] to a new String[] - then not used by default attribute provider.
      2. Subaccess checks are insanely expensive. The full path of every subdir is created by walking up the inode tree, creating a INode[], building a string by converting each inode's byte[] name to a string, etc. Which will only be used if there's an exception.

      The expensive of #1 should only be incurred when using the provider/enforcer feature. For #2, paths should be created on-demand for exceptions.

      Attachments

        1. HDFS-10673.patch
          18 kB
          Daryn Sharp
        2. HDFS-10673.1.patch
          21 kB
          Daryn Sharp
        3. HDFS-10673.2.patch
          20 kB
          Daryn Sharp
        4. HDFS-10673-branch-2.7.00.patch
          21 kB
          Zhe Zhang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            daryn Daryn Sharp Assign to me
            daryn Daryn Sharp
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment