Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13136

Avoid taking FSN lock while doing group member lookup for FSD permission check

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0, 3.2.0, 3.0.3
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Namenode has FSN lock and FSD lock. Most of the namenode operations need to take FSN lock first and then FSD lock. The permission check is done via FSPermissionChecker at FSD layer assuming FSN lock is taken.

      The FSPermissionChecker constructor invokes callerUgi.getGroups() that can take seconds sometimes. There are external cache scheme such SSSD and internal cache scheme for group lookup. However, the delay could still occur during cache refresh, which causes severe FSN lock contentions and unresponsive namenode issues.

      Checking the current code, we found that getBlockLocations(..) did it right but some methods such as getFileInfo(..), getContentSummary(..) did it wrong. This ticket is open to ensure the group lookup for permission checker is outside the FSN lock.

        Attachments

        1. HDFS-13136-branch-3.0.002.patch
          73 kB
          Xiaoyu Yao
        2. HDFS-13136-branch-3.0.001.patch
          72 kB
          Xiaoyu Yao
        3. HDFS-13136.002.patch
          69 kB
          Xiaoyu Yao
        4. HDFS-13136.001.patch
          67 kB
          Xiaoyu Yao

          Activity

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              xyao Xiaoyu Yao
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: