Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12782

Faster LDAP group name resolution with ActiveDirectory



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: security
    • Labels:
    • Hadoop Flags:
    • Release Note:
      If the user object returned by LDAP server has the user's group object DN (supported by Active Directory), Hadoop can reduce LDAP group mapping latency by setting hadoop.security.group.mapping.ldap.search.attr.memberof to memberOf.


      The typical LDAP group name resolution works well under typical scenarios. However, we have seen cases where a user is mapped to many groups (in an extreme case, a user is mapped to more than 100 groups). The way it's being implemented now makes this case super slow resolving groups from ActiveDirectory.

      The current LDAP group resolution implementation sends two queries to a ActiveDirectory server. The first query returns a user object, which contains DN (distinguished name). The second query looks for groups where the user DN is a member. If a user is mapped to many groups, the second query returns all group objects associated with the user, and is thus very slow.

      After studying a user object in ActiveDirectory, I found a user object actually contains a "memberOf" field, which is the DN of all group objects where the user belongs to. Assuming that an organization has no recursive group relation (that is, a user A is a member of group G1, and group G1 is a member of group G2), we can use this properties to avoid the second query, which can potentially run very slow.

      I propose that we add a configuration to only enable this feature for users who want to reduce group resolution time and who does not have recursive groups, so that existing behavior will not be broken.


        1. HADOOP-12782.branch-2.010.patch
          31 kB
          Wei-Chiu Chuang
        2. HADOOP-12782.009.patch
          31 kB
          Wei-Chiu Chuang
        3. HADOOP-12782.008.patch
          31 kB
          Wei-Chiu Chuang
        4. HADOOP-12782.007.patch
          26 kB
          Wei-Chiu Chuang
        5. HADOOP-12782.006.patch
          21 kB
          Wei-Chiu Chuang
        6. HADOOP-12782.005.patch
          21 kB
          Wei-Chiu Chuang
        7. HADOOP-12782.004.patch
          21 kB
          Wei-Chiu Chuang
        8. HADOOP-12782.003.patch
          21 kB
          Wei-Chiu Chuang
        9. HADOOP-12782.002.patch
          16 kB
          Wei-Chiu Chuang
        10. HADOOP-12782.001.patch
          17 kB
          Wei-Chiu Chuang

          Issue Links



              • Assignee:
                weichiu Wei-Chiu Chuang
                weichiu Wei-Chiu Chuang
              • Votes:
                0 Vote for this issue
                11 Start watching this issue


                • Created: