Hadoop Common
  1. Hadoop Common
  2. HADOOP-6855

Add ability to get groups for ACLs from 'getent netgroup'

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.20.3
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Add ability to specify netgroups in ACLs (see class AccessControlList.java). Membership of users in netgroups will be determined by running 'getent negroups $groupName'. Netgroups will be differentiated from regular unix groups by having '@' as a first character.

      1. HADOOP-6855-0.20-2.patch
        19 kB
        Erik Steffl
      2. HADOOP-6855-0.20-1.patch
        19 kB
        Erik Steffl
      3. HADOOP-6855-0.20.patch
        19 kB
        Erik Steffl

        Issue Links

          Activity

          Hide
          Boris Shkolnik added a comment -

          Here are some comments so far..
          1. commented log lines Shellbasedunixgroup:122 and after could be removed

          2. getNetgroups - returns list that is passed to it as an argument. Do you need this return value?

          3. List groups = new LinkedList(netgroupToUsersMap.keySet()); - shouldn't it be a generic LinkedList<E>.

          4. ShellBasedUnixGroupsNetgroupMappingTestWrapper: execShellGetUnixGroups - seems to return wrong data - call for groups for user foo doesn't return user name as the first entry (not sure if it is used in tests though).

          5. Patch seems to introduce 2 new warnings:
          ShellBasedUnixGroupsNetgroupMapping.java:60: warning: [unchecked] unchecked call to Lin
          kedList(java.util.Collection<? extends E>) as a member of the raw type java.util.LinkedList
          [javac] List groups = new LinkedList(netgroupToUsersMap.keySet());
          [javac] ^
          [javac] ShellBasedUnixGroupsNetgroupMapping.java:62: warning: [unchecked] unchecked conversion
          [javac] found : java.util.List
          [javac] required: java.util.List<java.lang.String>
          [javac] cacheGroupsAdd(groups);
          [javac] ^

          Show
          Boris Shkolnik added a comment - Here are some comments so far.. 1. commented log lines Shellbasedunixgroup:122 and after could be removed 2. getNetgroups - returns list that is passed to it as an argument. Do you need this return value? 3. List groups = new LinkedList(netgroupToUsersMap.keySet()); - shouldn't it be a generic LinkedList<E>. 4. ShellBasedUnixGroupsNetgroupMappingTestWrapper: execShellGetUnixGroups - seems to return wrong data - call for groups for user foo doesn't return user name as the first entry (not sure if it is used in tests though). 5. Patch seems to introduce 2 new warnings: ShellBasedUnixGroupsNetgroupMapping.java:60: warning: [unchecked] unchecked call to Lin kedList(java.util.Collection<? extends E>) as a member of the raw type java.util.LinkedList [javac] List groups = new LinkedList(netgroupToUsersMap.keySet()); [javac] ^ [javac] ShellBasedUnixGroupsNetgroupMapping.java:62: warning: [unchecked] unchecked conversion [javac] found : java.util.List [javac] required: java.util.List<java.lang.String> [javac] cacheGroupsAdd(groups); [javac] ^
          Hide
          Erik Steffl added a comment -

          HADOOP-6855-0.20-1.patch fixes problems pointed out by review:

          1. ShellBasedUnixGroupsNetgroupMapping.java: commented out log lines removed

          2. ShellBasedUnixGroupsNetgroupMapping.java: getNetgroups, getUnixGroups return void now

          3. ShellBasedUnixGroupsNetgroupMapping.java: LinkedList<String> is used

          4. ShellBasedUnixGroupsNetgroupMappingTestWrapper.java: yes, the first element was user, removed. And yes, it doesn't really matter for tests

          5. No warnings, was caused by usage of LinkedList instead of LinkedList<String>

          Also, per our discussion, I improved parsing of netgroup output, previously supported format:

          group ( , user, ) ...

          Currently supported format (both domain and host are optional, not used if present):

          group ( , user, ) ( domain, anotherUser, host.com)

          Show
          Erik Steffl added a comment - HADOOP-6855 -0.20-1.patch fixes problems pointed out by review: 1. ShellBasedUnixGroupsNetgroupMapping.java: commented out log lines removed 2. ShellBasedUnixGroupsNetgroupMapping.java: getNetgroups, getUnixGroups return void now 3. ShellBasedUnixGroupsNetgroupMapping.java: LinkedList<String> is used 4. ShellBasedUnixGroupsNetgroupMappingTestWrapper.java: yes, the first element was user, removed. And yes, it doesn't really matter for tests 5. No warnings, was caused by usage of LinkedList instead of LinkedList<String> Also, per our discussion, I improved parsing of netgroup output, previously supported format: group ( , user, ) ... Currently supported format (both domain and host are optional, not used if present): group ( , user, ) ( domain, anotherUser, host.com)
          Hide
          Boris Shkolnik added a comment -

          +1

          small nit, there was one more commented line which is not needed (// netgroupToUsersMap.put(group, users); )

          Show
          Boris Shkolnik added a comment - +1 small nit, there was one more commented line which is not needed (// netgroupToUsersMap.put(group, users); )
          Hide
          Erik Steffl added a comment -

          Patch HADOOP-6855-0.20-2.patch removes the bogus comment (// netgroupToUsersMap.put(group, users); )

          Show
          Erik Steffl added a comment - Patch HADOOP-6855 -0.20-2.patch removes the bogus comment (// netgroupToUsersMap.put(group, users); )
          Hide
          Erik Steffl added a comment -

          Testing: ant test fails with the same errors with or without patch.

          Show
          Erik Steffl added a comment - Testing: ant test fails with the same errors with or without patch.
          Hide
          Allen Wittenauer added a comment -

          -1

          getent is not portable.

          Show
          Allen Wittenauer added a comment - -1 getent is not portable.
          Hide
          Erik Steffl added a comment -

          Re https://issues.apache.org/jira/browse/HADOOP-6855?focusedCommentId=12889373&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12889373

          This patch is adding an optional group to users mapping mechanism for users who want to use getent (often used to get group to users mapping from LDAP). Users who want unix group mapping (itself not entirely portable) can continue to use existing mechanism of group to users mapping.

          Show
          Erik Steffl added a comment - Re https://issues.apache.org/jira/browse/HADOOP-6855?focusedCommentId=12889373&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12889373 This patch is adding an optional group to users mapping mechanism for users who want to use getent (often used to get group to users mapping from LDAP). Users who want unix group mapping (itself not entirely portable) can continue to use existing mechanism of group to users mapping.
          Hide
          Allen Wittenauer added a comment -

          LDAP and netgroups are portable. For netgroups, most Unices have getnetgrent() and friends. For LDAP, you can query it natively from Java with JNDI.

          The problem is the getent command itself. It is a SysV command, so many BSD's do not have it, including OS X.

          Show
          Allen Wittenauer added a comment - LDAP and netgroups are portable. For netgroups, most Unices have getnetgrent() and friends. For LDAP, you can query it natively from Java with JNDI. The problem is the getent command itself. It is a SysV command, so many BSD's do not have it, including OS X.
          Hide
          Erik Steffl added a comment -

          This patch is for people who want to use 'getent netgroup $group' command to provide groups to user mapping. It does not affect people who do not explicitly configure usage of the group mapping service added in this patch (ShellBasedUnixGroupsNetgroupMapping).

          There is another patch coming that will provide JNI (or JNA) based implementation see https://issues.apache.org/jira/browse/HADOOP-6864

          Show
          Erik Steffl added a comment - This patch is for people who want to use 'getent netgroup $group' command to provide groups to user mapping. It does not affect people who do not explicitly configure usage of the group mapping service added in this patch (ShellBasedUnixGroupsNetgroupMapping). There is another patch coming that will provide JNI (or JNA) based implementation see https://issues.apache.org/jira/browse/HADOOP-6864
          Hide
          Erik Steffl added a comment -

          This patch is not to be committed, there is a separate jira for trunk (0.23 at the moment) that is more complete, it also includes JNI implementation of the netgroups support, see:

          https://issues.apache.org/jira/browse/HADOOP-6864

          Show
          Erik Steffl added a comment - This patch is not to be committed, there is a separate jira for trunk (0.23 at the moment) that is more complete, it also includes JNI implementation of the netgroups support, see: https://issues.apache.org/jira/browse/HADOOP-6864

            People

            • Assignee:
              Erik Steffl
              Reporter:
              Erik Steffl
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development