Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13988

KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.3
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.4
    • Component/s: common, kms
    • Labels:
      None
    • Environment:

      HDP 2.5.3.0

      WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes

    • Hadoop Flags:
      Reviewed

      Description

      After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.

      So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:

           }
           // Use current user by default
           UserGroupInformation actualUgi = currentUgi;
           if (currentUgi.getRealUser() != null) {
             // Use real user for proxy user
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using RealUser for proxyUser);
      	}
             actualUgi = currentUgi.getRealUser();
             if (getDoAsUser() != null) {
            	  if (LOG.isDebugEnabled()) {
      		LOG.debug("doAsUser exists");
      		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
      		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
                }
          	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
          		  if (LOG.isDebugEnabled()) {
      		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
      		  }
      		  actualUgi = UserGroupInformation.getLoginUser();
      		  if (LOG.isDebugEnabled()) {
      	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
      		  }
           	  }
             }
      	
           } else if (!currentUgiContainsKmsDt() &&
               !currentUgi.hasKerberosCredentials()) {
             // Use login user for user that does not have either
             // Kerberos credential or KMS delegation token for KMS operations
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
      	}
             actualUgi = currentUgi.getLoginUser();
           }
           return actualUgi;
         }
      
      

        Attachments

        1. HADOOP-13988.patch
          2 kB
          Greg Senia
        2. HADOOP-13988.patch
          3 kB
          Greg Senia
        3. HADOOP-13988.03.patch
          3 kB
          Xiaoyu Yao
        4. HADOOP-13988.02.patch
          2 kB
          Xiaoyu Yao
        5. HADOOP-13988.01.patch
          2 kB
          Xiaoyu Yao

          Issue Links

            Activity

              People

              • Assignee:
                xyao Xiaoyu Yao
                Reporter:
                gss2002 Greg Senia
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: