[KNOX-762] Remove dependency on httpcomponents httpclient 4.5.2

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: Server
• External issue URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1712
• Assignee: Larry McCay
• Reporter: Larry McCay
• Created: 21/Oct/16 14:26
• Updated: 28/Mar/19 14:17
• Resolved: 21/Oct/16 16:31

Description

Reported by Benjamin Ruland:

I am experiencing problems with Knox while using WebHDFS in a cluster with Kerberos and SSL.
The KDC is an Microsoft AD 2012. Kerberos-Encryption is set to AES256. Knox is connected to AD via LDAP sync (this is working fine for other Knox services).
I am running HDP 2.5 with Knox 0.9.0

In general, the cluster runs fine. WebHDFS using SPNEGO is working.

But when accessing WebHDFS over Knox, I get an 401 error and some strange logs.
I suspect that Knox is trying to get a ticket for a HTTPS/namenode@REALM principal, which does not exist. Although running SSL, all principals for SPNEGO are HTTP/...

I this a Knox Bug or is this a misconfiguration at some point?

It would be great, if someone has advice.

Best regards,
Benjamin

The used command is:

[root@utilitynode ~]# curl -ik -u validuser "https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS"
Enter host password for user 'validuser':
HTTP/1.1 401 Unauthorized
Date: Wed, 12 Oct 2016 07:47:41 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Tue,11-Oct-2016 07:47:41 GMT
WWW-Authenticate: BASIC realm="application"
Content-Length: 0
Server: Jetty(9.2.15.v20160210)

Debug Log in knox gateway.log

2016-10-12 09:51:49,735 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1/
2016-10-12 09:51:49,740 DEBUG hadoop.gateway (KnoxLdapRealm.java:getUserDn(673)) - Searching from OU=someOU,DC=somedomain,DC=de where (&(objectclass=person)(sAMAccountName=validuser)) scope subtree
2016-10-12 09:51:49,745 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(679)) - Computed userDn: CN=validuser,OU=Users,OU=someOU,DC=somedomain,DC=de using ldapSearch for principal: validuser
2016-10-12 09:51:49,749 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS, direction: IN via explicit rule: WEBHDFS/webhdfs/inbound/namenode/root to URL: https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS
2016-10-12 09:51:49,749 DEBUG hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(120)) - Dispatch request: GET https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS&doAs=validuser
2016-10-12 09:51:49,781 WARN auth.HttpAuthenticator (HttpAuthenticator.java:generateAuthResponse(207)) - NEGOTIATE authentication error: No valid credentials provided (Mechanism level: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7)))
2016-10-12 09:51:49,782 DEBUG hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(133)) - Dispatch response status: 401
2016-10-12 09:51:49,783 DEBUG hadoop.gateway (DefaultDispatch.java:getInboundResponseContentType(202)) - Using explicit character set ISO-8859-1 for entity of type text/html
2016-10-12 09:51:49,783 DEBUG hadoop.gateway (DefaultDispatch.java:getInboundResponseContentType(210)) - Inbound response entity content type: text/html; charset=iso-8859-1

Log in knox gateway.out

Found ticket for knox/[e-mail] to go to krbtgt/[e-mail] expiring on Wed Oct 12 19:53:51 CEST 2016
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
default etypes for default_tgs_enctypes: 18.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=domaincontroller.somedomain.de. TCP:88, timeout=30000, number of retries =3, #bytes=1661
>>> KDCCommunication: kdc=domaincontroller.somedomain.de. TCP:88, timeout=30000,Attempt =1, #bytes=1661
>>>DEBUG: TCPClient reading 127 bytes
>>> KrbKdcReq send: #bytes read=127
>>> KdcAccessibility: remove domaincontroller.somedomain.de.:88
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
sTime is Wed Oct 12 09:53:51 CEST 2016 1476258831000
suSec is 8354 suSec is 8354
error code is 7
error Message is Server not found in Kerberos database
sname is HTTPS/[e-mail]
msgType is 30

Extracts from topology config:

<topology>

<gateway>

<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>

<!-- LDAP Sync properties sit here -->

<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>

<provider>
<role>authorization</role>
<name>XASecurePDPKnox</name>
<enabled>true</enabled>
</provider>

<provider>
<role>ha</role>
<name>HaProvider</name>
<enabled>true</enabled>
<param>
<name>WEBHDFS</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;maxRetryAttempts=300;retrySleep=1000;enabled=true</value>
</param>
</provider>

</gateway>

<service>
<role>NAMENODE</role>
<url>hdfs://namenode.somedomain.de:8020</url>
<url>hdfs://namenode2.somedomain.de:8020</url>
</service>

<service>
<role>WEBHDFS</role>
<url>https://namenode.somedomain.de:50470/webhdfs</url>
<url>https://namenode2.somedomain.de:50470/webhdfs</url>
</service>

</topology>

Issue Links

• relates to: KNOX-904 4.5.2 httpclient dependency re-introduced