Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13988

KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.3
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: common, kms
    • Labels:
    • Environment:


      WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes

    • Hadoop Flags:


      After upgrading to HDP noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.

      So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:

           // Use current user by default
           UserGroupInformation actualUgi = currentUgi;
           if (currentUgi.getRealUser() != null) {
             // Use real user for proxy user
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using RealUser for proxyUser);
             actualUgi = currentUgi.getRealUser();
             if (getDoAsUser() != null) {
            	  if (LOG.isDebugEnabled()) {
      		LOG.debug("doAsUser exists");
      		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
      		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
          	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
          		  if (LOG.isDebugEnabled()) {
      		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
      		  actualUgi = UserGroupInformation.getLoginUser();
      		  if (LOG.isDebugEnabled()) {
      	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
           } else if (!currentUgiContainsKmsDt() &&
               !currentUgi.hasKerberosCredentials()) {
             // Use login user for user that does not have either
             // Kerberos credential or KMS delegation token for KMS operations
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
             actualUgi = currentUgi.getLoginUser();
           return actualUgi;


        1. HADOOP-13988.patch
          2 kB
          Greg Senia
        2. HADOOP-13988.patch
          3 kB
          Greg Senia
        3. HADOOP-13988.01.patch
          2 kB
          Xiaoyu Yao
        4. HADOOP-13988.02.patch
          2 kB
          Xiaoyu Yao
        5. HADOOP-13988.03.patch
          3 kB
          Xiaoyu Yao

          Issue Links



              • Assignee:
                xyao Xiaoyu Yao
                gss2002 Greg Senia
              • Votes:
                0 Vote for this issue
                9 Start watching this issue


                • Created: