Uploaded image for project: 'Apache Knox'
  1. Apache Knox
  2. KNOX-762

Remove dependency on httpcomponents httpclient 4.5.2



    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.10.0
    • Server
    • None


      Reported by Benjamin Ruland:

      I am experiencing problems with Knox while using WebHDFS in a cluster with Kerberos and SSL.
      The KDC is an Microsoft AD 2012. Kerberos-Encryption is set to AES256. Knox is connected to AD via LDAP sync (this is working fine for other Knox services).
      I am running HDP 2.5 with Knox 0.9.0

      In general, the cluster runs fine. WebHDFS using SPNEGO is working.

      But when accessing WebHDFS over Knox, I get an 401 error and some strange logs.
      I suspect that Knox is trying to get a ticket for a HTTPS/namenode@REALM principal, which does not exist. Although running SSL, all principals for SPNEGO are HTTP/...

      I this a Knox Bug or is this a misconfiguration at some point?

      It would be great, if someone has advice.

      Best regards,

      The used command is:

      [root@utilitynode ~]# curl -ik -u validuser "https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS"
      Enter host password for user 'validuser':
      HTTP/1.1 401 Unauthorized
      Date: Wed, 12 Oct 2016 07:47:41 GMT
      Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Tue,11-Oct-2016 07:47:41 GMT
      WWW-Authenticate: BASIC realm="application"
      Content-Length: 0
      Server: Jetty(9.2.15.v20160210)

      Debug Log in knox gateway.log

      2016-10-12 09:51:49,735 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1/
      2016-10-12 09:51:49,740 DEBUG hadoop.gateway (KnoxLdapRealm.java:getUserDn(673)) - Searching from OU=someOU,DC=somedomain,DC=de where (&(objectclass=person)(sAMAccountName=validuser)) scope subtree
      2016-10-12 09:51:49,745 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(679)) - Computed userDn: CN=validuser,OU=Users,OU=someOU,DC=somedomain,DC=de using ldapSearch for principal: validuser
      2016-10-12 09:51:49,749 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS, direction: IN via explicit rule: WEBHDFS/webhdfs/inbound/namenode/root to URL: https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS
      2016-10-12 09:51:49,749 DEBUG hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(120)) - Dispatch request: GET https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS&doAs=validuser
      2016-10-12 09:51:49,781 WARN auth.HttpAuthenticator (HttpAuthenticator.java:generateAuthResponse(207)) - NEGOTIATE authentication error: No valid credentials provided (Mechanism level: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7)))
      2016-10-12 09:51:49,782 DEBUG hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(133)) - Dispatch response status: 401
      2016-10-12 09:51:49,783 DEBUG hadoop.gateway (DefaultDispatch.java:getInboundResponseContentType(202)) - Using explicit character set ISO-8859-1 for entity of type text/html
      2016-10-12 09:51:49,783 DEBUG hadoop.gateway (DefaultDispatch.java:getInboundResponseContentType(210)) - Inbound response entity content type: text/html; charset=iso-8859-1

      Log in knox gateway.out

      Found ticket for knox/utilitynode.somedomain.de@SOMEDOMAIN.DE to go to krbtgt/somedomain.de@SOMEDOMAIN.DE expiring on Wed Oct 12 19:53:51 CEST 2016
      Entered Krb5Context.initSecContext with state=STATE_NEW
      Service ticket not found in the subject
      >>> Credentials acquireServiceCreds: same realm
      default etypes for default_tgs_enctypes: 18.
      >>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
      >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
      getKDCFromDNS using UDP
      >>> KrbKdcReq send: kdc=domaincontroller.somedomain.de. TCP:88, timeout=30000, number of retries =3, #bytes=1661
      >>> KDCCommunication: kdc=domaincontroller.somedomain.de. TCP:88, timeout=30000,Attempt =1, #bytes=1661
      >>>DEBUG: TCPClient reading 127 bytes
      >>> KrbKdcReq send: #bytes read=127
      >>> KdcAccessibility: remove domaincontroller.somedomain.de.:88
      >>> KDCRep: init() encoding tag is 126 req type is 13
      sTime is Wed Oct 12 09:53:51 CEST 2016 1476258831000
      suSec is 8354 suSec is 8354
      error code is 7
      error Message is Server not found in Kerberos database
      sname is HTTPS/namenode.somedomain.de@SOMEDOMAIN.DE
      msgType is 30

      Extracts from topology config:




      <!-- LDAP Sync properties sit here -->









        Issue Links



              lmccay Larry McCay
              lmccay Larry McCay
              0 Vote for this issue
              3 Start watching this issue