Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 2.6.0
-
Fix Version/s: 2.8.0, 3.0.0-alpha1
-
Component/s: security
-
Labels:
-
Target Version/s:
Description
Pre-JDK8, GSSName auto-canonicalized the hostname when constructing the principal for SPNEGO. JDK8 no longer does this which breaks the use of user-friendly CNAMEs for services.
Issue Links
- duplicates
-
HADOOP-11888
bootstrapStandby command broken in JDK1.8 with kerberos
-
- Resolved
-
- Is contained by
-
BIGTOP-2673
Need to back port HADOOP-11628 for JDK8
-
- Closed
-
- is depended upon by
-
HADOOP-11090
[Umbrella] Support Java 8 in Hadoop
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
I don't think this is the correct fix. I'm fairly certain that SPNs that are CNAMEs are supposed to stay CNAMEs. In other words, JDK8 fixed JDK7's broken behavior.
OK, it looks like it is implementation dependent. MIT does canonicalize, MS does not. Wheeeeeee.....
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12700519/HADOOP-11628.patch
against trunk revision 1aea440.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-auth.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5767//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5767//console
This message is automatically generated.
Pretty much all browsers and cmdline tools like curl default to canonicalization. I don't have access to Windows hosts but pretty sure it does the same.
Win 2k does, Win 2k3 does not, based upon https://technet.microsoft.com/en-us/library/cc772815%28v=ws.10%29.aspx .
Ugh: http://stackoverflow.com/questions/12229658/java-spnego-unwanted-spn-canonicalization
If you follow the thread mentioned, gives more details on why this is a bad deployment strategy. So I'm thinking this should probably be a runtime option with a default of off.
If you follow the thread mentioned, gives more details on why this is a bad deployment strategy. So I'm thinking this should probably be a runtime option with a default of off.
Only argument against canonicalization is trusting DNS. One could argue that as site that wouldn't trust its DNS for reverse lookup should have similar reservations against forward lookups as well.
Canonicalization (or a way to append default domain) is required to support short names in service URIs as well. GSSAPI (rfc2743) and Kerberos 5 (rfc4120) are not specific to SPNEGO, which is specific to http, where there is a provision to provide Host header. GSSAPI auth with ssh against multi-a rotation has same challenges. NN<>DN negotiate spn and validated against allowed list in configuration (dfs.namenode.kerberos.principal.pattern)
I agree this wouldn't work across all deployment strategies (eg: using Akami for failover/load balancing) and should be configurable and should be documented as to how clients and servers are expected to construct service principle.
10+ year old software isn't a consideration. All that matters is: what do today's browsers and user agents do? Answer: canonicalize. Years ago, windows firefox and some old versions of chrome didn't canonicalize so they didn't work with cnames because the de facto standard was canonicalize. Now they all do.
Pre-jdk8 automatically canonicalized. Hadoop + spnego + cnames worked. Now it doesn't. In jdk8, as an app developer, you have the option to not require canonicalization. However for spnego I question the need for yet another conf key (YACK!) to add previously unavailable behavior (not canonicalize), which if used will break every browser.
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 14m 32s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
| +1 | javac | 7m 27s | There were no new javac warning messages. |
| +1 | javadoc | 9m 32s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. |
| +1 | checkstyle | 0m 20s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 33s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 0m 34s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. |
| +1 | common tests | 5m 17s | Tests passed in hadoop-auth. |
| 40m 14s |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12700519/HADOOP-11628.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1b3b9e5 |
| hadoop-auth test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/6236/artifact/patchprocess/testrun_hadoop-auth.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/6236/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/6236/console |
This message was automatically generated.
Bumping priority because this "regression" in the JDK's behavior is unacceptable for SPNEGO in production environments.
Daryn Sharp, we haven't officially started supporting JDK 8 yet (HADOOP-11090), that is why I moved this to 2.8.0. Any reason why this is in 2.7.1?
Thinking out aloud, should we put this in 2.8.0 when we will have HADOOP-11090 with a config option on by default for older deployments?
IAC, Allen Wittenauer / Rajiv Chittajallu, please respond to Daryn Sharp's comment above. Let's get to some consensus here. Tx.
Canceling patch as the discussion stalled.
Moving it back out to 2.8.0 per my previous comment. Please revert back if you disagree.
OK: is there any reason why the patch as is SHOULD NOT go into 2.8?
Regarding options, SPNEGO isn't that broadly used in the Hadoop stack at least with Jersey (KMS, WebHDFS & Timeline, each with their own client). I do want to coalesce these with a single common HTTP/Jersey client. Having this feature in without another config option would work better on Java 8, and avoid adding another dimension to the configuration-space which is hadoop's -site.xml and the tests around it. Assuming this is a no-op on Java 7, enabling it will give consistent behaviour for Java 8, so it should not count as a regression there
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 18m 36s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
| +1 | javac | 9m 10s | There were no new javac warning messages. |
| +1 | javadoc | 11m 32s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 25s | The applied patch does not increase the total number of release audit warnings. |
| +1 | checkstyle | 0m 24s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 38s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 0m 47s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | common tests | 13m 40s | Tests passed in hadoop-auth. |
| 56m 52s |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12700519/HADOOP-11628.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 88d89267 |
| hadoop-auth test log | https://builds.apache.org/job/PreCommit-HADOOP-Build/7678/artifact/patchprocess/testrun_hadoop-auth.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/7678/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/7678/console |
This message was automatically generated.
+1, ideally with a backport to 2.7.x branch
FAILURE: Integrated in Hadoop-trunk-Commit #8655 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8655/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
- hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Yarn-trunk #1282 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1282/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #561 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/561/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #546 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/546/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-common/CHANGES.txt
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2495 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2495/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
- hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #509 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/509/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
- hadoop-common-project/hadoop-common/CHANGES.txt
FAILURE: Integrated in Hadoop-Hdfs-trunk #2446 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2446/)
HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn (stevel: rev bafeb6c7bc50efd11c6637921a50dd9cfdd53841)
- hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/KerberosAuthenticationHandler.java
- hadoop-common-project/hadoop-common/CHANGES.txt
+1, ideally with a backport to 2.7.x branch
Thanks Steve! Patch seems to apply cleanly to branch-2.7 (but I'm infamous for screwing up git). Did you encounter a problem?
Revisiting this: this isn't in 2.7... do we need to backport it?
Explicitly canonicalize. Cannot test due to inability to fake cnames, but has been tested internally.