Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 2.7.0, 2.8.0
-
Fix Version/s: 2.8.0, 3.0.0-alpha1
-
Component/s: security
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
In HADOOP-9477, posixGroup support was added
In HADOOP-10626, a limit on the returned attributes was added to speed up queries.
Limiting the attributes can break the SEARCH_CONTROLS object in the context of the isPosix block, since it only asks LDAP for the groupNameAttr
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 14m 45s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
| +1 | javac | 7m 31s | There were no new javac warning messages. |
| +1 | javadoc | 9m 36s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 6s | The applied patch generated 3 new checkstyle issues (total was 33, now 35). |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 36s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 1m 40s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | common tests | 23m 54s | Tests passed in hadoop-common. |
| 61m 7s |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12734008/HADOOP-12001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7401e5b |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/6758/artifact/patchprocess/diffcheckstylehadoop-common.txt |
| hadoop-common test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6758/artifact/patchprocess/testrun_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6758/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6758/console |
This message was automatically generated.
Bumping this to be a blocker for 2.7.1 given Elliot Clark's comment on the mailing lists
HADOOP-12001should probably be added to the blocker list since it's a
regression that can keep ldap from working.
Assigning this to Patrick White who put up a patch.
Given HADOOP-9477 was only in 2.8, the only breaking change in 2.7 is HADOOP-10626?
Those two patches on their own don't break anything, but the combination will break posixGroups implementation as it stands.
My patch corrects that.
I'm new to patches in Hadoop, what additional work is required on the patch?
For testing, I've run it on a cluster and verified I see the right thing when configured in posixGroups mode with the 'groups' command. I'm not 100% sure how to modify the unit tests to fit this case, but haven't looked very deeply yet.
I'm new to patches in Hadoop, what additional work is required on the patch?
For testing, I've run it on a cluster and verified I see the right thing when configured in posixGroups mode with the 'groups' command. I'm not 100% sure how to modify the unit tests to fit this case, but haven't looked very deeply yet.
I looked at the patch. Seems compatible, that part looks good. Can you see TestLdapGroupsMapping and TestLdapGroupsMappingWithPosixGroup and look if you can add a test there?
same patch with updated unit tests
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 16m 18s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | javac | 7m 29s | There were no new javac warning messages. |
| +1 | javadoc | 9m 33s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 5s | The applied patch generated 3 new checkstyle issues (total was 33, now 35). |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 35s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 1m 49s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | common tests | 22m 57s | Tests passed in hadoop-common. |
| 61m 43s |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12737380/HADOOP-12001.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / bc85959 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/6908/artifact/patchprocess/diffcheckstylehadoop-common.txt |
| hadoop-common test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6908/artifact/patchprocess/testrun_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6908/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6908/console |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 16m 32s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | javac | 7m 33s | There were no new javac warning messages. |
| +1 | javadoc | 9m 35s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 4s | The applied patch generated 3 new checkstyle issues (total was 33, now 35). |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 34s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 1m 49s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | common tests | 23m 48s | Tests passed in hadoop-common. |
| 62m 56s |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12737380/HADOOP-12001.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / bc85959 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/6909/artifact/patchprocess/diffcheckstylehadoop-common.txt |
| hadoop-common test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6909/artifact/patchprocess/testrun_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6909/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6909/console |
This message was automatically generated.
+1 (non-binding) from me. Everything looks good here.
Looks good to me too. Checking this in..
Given
HADOOP-9477was only in 2.8, the only breaking change in 2.7 isHADOOP-10626?
Given the above, I only put this in 2.8.
Committed this to trunk and branch-2. Thanks, Patrick!
FAILURE: Integrated in Hadoop-trunk-Commit #8018 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8018/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/CHANGES.txt
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Yarn-trunk #960 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/960/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/230/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Hdfs-trunk #2158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2158/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #219 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/219/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/228/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/CHANGES.txt
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2176 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2176/)
HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. (vinodkv: rev 722aa1db1f2ac3db0e70063022436a90f90643f3)
- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithPosixGroup.java
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
HADOOP-12001. Moving CHANGES.txt up into 2.8. (vinodkv: rev 04c9a077c8f88e3d8ed3e7966966e13f68c5c2ec) - hadoop-common-project/hadoop-common/CHANGES.txt
Initial attempt at a fix for this.
schemas to use it.
Shouldn't break anything that currently uses this, but enables people with weird