[HADOOP-14743] CompositeGroupsMapping should not swallow exceptions - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 2.5.0
Fix Version/s: 2.9.0, 3.0.0-beta1
Component/s: security
Labels:
None

Target Version/s:

2.9.0, 3.0.0-beta1, 2.8.2, 2.7.5
Hadoop Flags:

Reviewed

Description

CompositeGroupsMapping#getGroups

   for (GroupMappingServiceProvider provider : providersList) {
      try {
        groups = provider.getGroups(user);
      } catch (Exception e) {
        //LOG.warn("Exception trying to get groups for user " + user, e);      
      }        
      if (groups != null && ! groups.isEmpty()) {
        groupSet.addAll(groups);
        if (!combined) break;
      }
    }

If anything fails inside the underlying groups mapping service provider, there's no way to tell what went wrong.

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

HADOOP-14743.001.patch

08/Aug/17 00:07

0.9 kB

Wei-Chiu Chuang
HADOOP-14743.002.patch

08/Aug/17 00:14

0.9 kB

Wei-Chiu Chuang

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Wei-Chiu Chuang added a comment - 08/Aug/17 00:08

Attach patch rev 001. This patch prints a warning message, and a more detailed stacktrace is printed at debug log level.

After this patch, the CompositeGroupsMapping would record the following log:

2017-08-07 17:05:31,696 WARN  security.CompositeGroupsMapping (CompositeGroupsMapping.java:getGroups(77)) - Unable to get groups for user Jack via UserProvider because: java.io.IOException: foo
2017-08-07 17:05:31,708 DEBUG security.CompositeGroupsMapping (CompositeGroupsMapping.java:getGroups(79)) - Stacktrace: 
java.io.IOException: foo
	at org.apache.hadoop.security.TestCompositeGroupMapping$UserProvider.getGroups(TestCompositeGroupMapping.java:120)
	at org.apache.hadoop.security.CompositeGroupsMapping.getGroups(CompositeGroupsMapping.java:75)
	at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384)
	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319)
	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
	at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)

Show

Wei-Chiu Chuang added a comment - 08/Aug/17 00:08 Attach patch rev 001. This patch prints a warning message, and a more detailed stacktrace is printed at debug log level. After this patch, the CompositeGroupsMapping would record the following log: 2017-08-07 17:05:31,696 WARN security.CompositeGroupsMapping (CompositeGroupsMapping.java:getGroups(77)) - Unable to get groups for user Jack via UserProvider because: java.io.IOException: foo 2017-08-07 17:05:31,708 DEBUG security.CompositeGroupsMapping (CompositeGroupsMapping.java:getGroups(79)) - Stacktrace: java.io.IOException: foo at org.apache.hadoop.security.TestCompositeGroupMapping$UserProvider.getGroups(TestCompositeGroupMapping.java:120) at org.apache.hadoop.security.CompositeGroupsMapping.getGroups(CompositeGroupsMapping.java:75) at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)

Hide

Permalink

Wei-Chiu Chuang added a comment - 08/Aug/17 00:14

Submit patch rev 002 to use the slf4j's curl bracket style parameters

Show

Wei-Chiu Chuang added a comment - 08/Aug/17 00:14 Submit patch rev 002 to use the slf4j's curl bracket style parameters

Hide

Permalink

Hadoop QA added a comment - 08/Aug/17 01:25

-1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0m 21s	Docker mode activated.
			Prechecks
+1	@author	0m 0s	The patch does not contain any @author tags.
-1	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			trunk Compile Tests
+1	mvninstall	17m 34s	trunk passed
+1	compile	14m 35s	trunk passed
+1	checkstyle	0m 36s	trunk passed
+1	mvnsite	1m 31s	trunk passed
+1	findbugs	1m 27s	trunk passed
+1	javadoc	0m 51s	trunk passed
			Patch Compile Tests
+1	mvninstall	0m 41s	the patch passed
+1	compile	11m 23s	the patch passed
+1	javac	11m 23s	the patch passed
+1	checkstyle	0m 37s	the patch passed
+1	mvnsite	1m 27s	the patch passed
+1	whitespace	0m 0s	The patch has no whitespace issues.
+1	findbugs	1m 36s	the patch passed
+1	javadoc	0m 51s	the patch passed
			Other Tests
+1	unit	8m 56s	hadoop-common in the patch passed.
+1	asflicense	0m 29s	The patch does not generate ASF License warnings.
		64m 46s

Subsystem	Report/Notes
Docker	Image:yetus/hadoop:14b5c93
JIRA Issue	~~HADOOP-14743~~
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12880745/HADOOP-14743.002.patch
Optional Tests	asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
uname	Linux e440ffadc2a7 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision	trunk / c61f2c4
Default Java	1.8.0_131
findbugs	v3.1.0-RC1
Test Results	https://builds.apache.org/job/PreCommit-HADOOP-Build/12974/testReport/
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://builds.apache.org/job/PreCommit-HADOOP-Build/12974/console
Powered by	Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

Show

Hadoop QA added a comment - 08/Aug/17 01:25 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. trunk Compile Tests +1 mvninstall 17m 34s trunk passed +1 compile 14m 35s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 31s trunk passed +1 findbugs 1m 27s trunk passed +1 javadoc 0m 51s trunk passed Patch Compile Tests +1 mvninstall 0m 41s the patch passed +1 compile 11m 23s the patch passed +1 javac 11m 23s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 1m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 36s the patch passed +1 javadoc 0m 51s the patch passed Other Tests +1 unit 8m 56s hadoop-common in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 64m 46s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14743 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880745/HADOOP-14743.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e440ffadc2a7 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c61f2c4 Default Java 1.8.0_131 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12974/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12974/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Hide

Permalink

Wei-Chiu Chuang added a comment - 08/Aug/17 16:28

Kai Zheng could you help review this patch? Thx

Show

Wei-Chiu Chuang added a comment - 08/Aug/17 16:28 Kai Zheng could you help review this patch? Thx

Hide

Permalink

Kai Zheng added a comment - 09/Aug/17 02:19

+1 for the nice change. Thanks!

Show

Kai Zheng added a comment - 09/Aug/17 02:19 +1 for the nice change. Thanks!

Hide

Permalink

Hudson added a comment - 10/Aug/17 17:02

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12160 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12160/)
~~HADOOP-14743~~. CompositeGroupsMapping should not swallow exceptions. (weichiu: rev a8b75466b21edfe8b12beb4420492817f0e03147)

(edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/CompositeGroupsMapping.java

Show

Hudson added a comment - 10/Aug/17 17:02 SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12160 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12160/ ) HADOOP-14743 . CompositeGroupsMapping should not swallow exceptions. (weichiu: rev a8b75466b21edfe8b12beb4420492817f0e03147) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/CompositeGroupsMapping.java

Hide

Permalink

Wei-Chiu Chuang added a comment - 10/Aug/17 17:10

Committed the last patch to trunk and branch-2. Thanks Kai Zheng for reviewing it!

Show

Wei-Chiu Chuang added a comment - 10/Aug/17 17:10 Committed the last patch to trunk and branch-2. Thanks Kai Zheng for reviewing it!

People

Assignee:

Wei-Chiu Chuang

Reporter:

Wei-Chiu Chuang

Votes:

0 Vote for this issue

Watchers:

4 Start watching this issue

Dates

Created:

08/Aug/17 00:01

Updated:

10/Aug/17 17:10

Resolved:

10/Aug/17 17:10

Development

Agile

View on Board