Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13988

KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0, 2.7.3
    • 2.9.0, 3.0.0-alpha4, 2.8.4
    • common, kms
    • None
    • HDP

      WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes

    • Reviewed


      After upgrading to HDP noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.

      So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:

           // Use current user by default
           UserGroupInformation actualUgi = currentUgi;
           if (currentUgi.getRealUser() != null) {
             // Use real user for proxy user
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using RealUser for proxyUser);
             actualUgi = currentUgi.getRealUser();
             if (getDoAsUser() != null) {
            	  if (LOG.isDebugEnabled()) {
      		LOG.debug("doAsUser exists");
      		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
      		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
          	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
          		  if (LOG.isDebugEnabled()) {
      		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
      		  actualUgi = UserGroupInformation.getLoginUser();
      		  if (LOG.isDebugEnabled()) {
      	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
           } else if (!currentUgiContainsKmsDt() &&
               !currentUgi.hasKerberosCredentials()) {
             // Use login user for user that does not have either
             // Kerberos credential or KMS delegation token for KMS operations
             if (LOG.isDebugEnabled()) {
      	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
             actualUgi = currentUgi.getLoginUser();
           return actualUgi;


        1. HADOOP-13988.01.patch
          2 kB
          Xiaoyu Yao
        2. HADOOP-13988.02.patch
          2 kB
          Xiaoyu Yao
        3. HADOOP-13988.03.patch
          3 kB
          Xiaoyu Yao
        4. HADOOP-13988.patch
          3 kB
          Greg Senia
        5. HADOOP-13988.patch
          2 kB
          Greg Senia

        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            xyao Xiaoyu Yao
            gss2002 Greg Senia
            0 Vote for this issue
            9 Start watching this issue




                Issue deployment