Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13137

TraceAdmin should support Kerberized cluster

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: tracing
    • Labels:
    • Environment:

      CDH5.5.1 cluster with Kerberos

    • Target Version/s:

      Description

      When I run hadoop trace command for a Kerberized NameNode, it failed with the following error:

      [hdfs@weichiu-encryption-1 root]$ hadoop trace -list -host weichiu-encryption-1.vpc.cloudera.com:802216/05/12 00:02:13 WARN ipc.Client: Exception encountered while connecting to the server : java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
      16/05/12 00:02:13 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs@VPC.CLOUDERA.COM (auth:KERBEROS) cause:java.io.IOException: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
      Exception in thread "main" java.io.IOException: Failed on local exception: java.io.IOException: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "weichiu-encryption-1.vpc.cloudera.com/172.26.8.185"; destination host is: "weichiu-encryption-1.vpc.cloudera.com":8022;
      at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
      at org.apache.hadoop.ipc.Client.call(Client.java:1470)
      at org.apache.hadoop.ipc.Client.call(Client.java:1403)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
      at com.sun.proxy.$Proxy11.listSpanReceivers(Unknown Source)
      at org.apache.hadoop.tracing.TraceAdminProtocolTranslatorPB.listSpanReceivers(TraceAdminProtocolTranslatorPB.java:58)
      at org.apache.hadoop.tracing.TraceAdmin.listSpanReceivers(TraceAdmin.java:68)
      at org.apache.hadoop.tracing.TraceAdmin.run(TraceAdmin.java:177)
      at org.apache.hadoop.tracing.TraceAdmin.main(TraceAdmin.java:195)
      Caused by: java.io.IOException: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
      at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:682)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:645)
      at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:733)
      at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
      at org.apache.hadoop.ipc.Client.getConnection(Client.java:1519)
      at org.apache.hadoop.ipc.Client.call(Client.java:1442)
      ... 7 more
      Caused by: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
      at org.apache.hadoop.security.SaslRpcClient.getServerPrincipal(SaslRpcClient.java:322)
      at org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:231)
      at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:159)
      at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:396)
      at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:555)
      at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:370)
      at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:725)
      at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:721)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:720)
      ... 10 more

      It is failing because TraceAdmin does not set up the property CommonConfigurationKeys.HADOOP_SECURITY_SERVICE_USER_NAME_KEY

      Fixing it may require some restructuring, as the NameNode principal dfs.namenode.kerberos.principal is a HDFS property, but TraceAdmin is in hadoop-common. Or, specify it with a new command -principal. Any suggestions? Thanks

        Attachments

        1. HADOOP-13137.005.patch
          7 kB
          Wei-Chiu Chuang
        2. HADOOP-13137.004.patch
          7 kB
          Wei-Chiu Chuang
        3. HADOOP-13137.003.patch
          7 kB
          Wei-Chiu Chuang
        4. HADOOP-13137.002.patch
          7 kB
          Wei-Chiu Chuang
        5. HADOOP-13137.001.patch
          2 kB
          Wei-Chiu Chuang

          Activity

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: