[HADOOP-17079] Optimize UGI#getGroups by adding UGI#getGroupsSet - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.4.0
Fix Version/s: 3.4.0
Component/s: build
Labels:
None

Target Version/s:

3.4.0
Hadoop Flags:

Reviewed
Release Note:

Hide
Added a UserGroupMapping#getGroupsSet() API and deprecate UserGroupMapping#getGroups.

The UserGroupMapping#getGroups() can be expensive as it involves Set->List conversion. For user with large group membership (i.e., > 1000 groups), we recommend using getGroupSet to avoid the conversion and fast membership look up.

Show
Added a UserGroupMapping#getGroupsSet() API and deprecate UserGroupMapping#getGroups. The UserGroupMapping#getGroups() can be expensive as it involves Set->List conversion. For user with large group membership (i.e., > 1000 groups), we recommend using getGroupSet to avoid the conversion and fast membership look up.

Description

UGI#getGroups has been optimized with ~~HADOOP-13442~~ by avoiding the List->Set->List conversion. However the returned list is not optimized to contains lookup, especially the user's group membership list is huge (thousands+) . This ticket is opened to add a UGI#getGroupsSet and use Set#contains() instead of List#contains() to speed up large group look up while minimize List->Set conversions in Groups#getGroups() call.