Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10890

ExportSnapshot needs to add acquired token to job

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.96.1, 0.98.1
    • 0.99.0, 0.98.2
    • snapshots
    • None
    • Reviewed

    Description

      ExportSnapshot failed when exporting to remote secure kerberos cluster

      biadmin@hdtest249:/opt/ibm/biginsights/hbase/lib> /opt/ibm/biginsights/hbase/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot s1 -copy-to hdfs://bdvm197.svl.ibm.com:9000/user/biadmin/mysnapshots
      14/04/01 14:07:20 INFO util.FSVisitor: No logs under directory:hdfs://hdtest249.svl.ibm.com:9000/hbase/.hbase-snapshot/s1/WALs
      14/04/01 14:07:22 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
      14/04/01 14:07:22 INFO compress.CodecPool: Got brand-new compressor [.deflate]
      14/04/01 14:07:22 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 17 for biadmin on 9.30.195.103:9000
      14/04/01 14:07:22 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 14 for biadmin on 9.30.122.206:9000
      14/04/01 14:07:24 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 18 for biadmin on 9.30.195.103:9000
      14/04/01 14:07:24 INFO security.TokenCache: Got dt for hdfs://hdtest249.svl.ibm.com:9000/user/biadmin/.staging/job_201404010039_0015/libjars/guava-12.0.1.jar;uri=9.30.195.103:9000;t.service=9.30.195.103:9000
      14/04/01 14:07:24 INFO input.FileInputFormat: Total input paths to process : 1
      14/04/01 14:07:25 INFO mapred.JobClient: Running job: job_201404010039_0015
      14/04/01 14:07:26 INFO mapred.JobClient:  map 0% reduce 0%
      14/04/01 14:07:41 INFO mapred.JobClient: Task Id : attempt_201404010039_0015_m_000000_0, Status : FAILED
      java.io.IOException: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "hdtest250.svl.ibm.com/9.30.195.104"; destination host is: "bdvm197.svl.ibm.com":9000;
       
      Caused by: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
              at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:173)
              at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:390)
              at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:494)
              at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:314)
              at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:659)
              at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:655)
              at java.security.AccessController.doPrivileged(AccessController.java:366)
              at javax.security.auth.Subject.doAs(Subject.java:572)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
              at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:654)
              ... 31 more
      

      With HBASE-10608, we acquired the tokens, but we didn't add them to the MR job.

      Attachments

        1. 10890-obtain-tokens.txt
          1 kB
          Ted Yu
        2. HBASE-10890-trunk.patch
          0.9 kB
          Jerry He
        3. HBASE-10890-trunk-v2.patch
          3 kB
          Jerry He

        Activity

          People

            jinghe Jerry He
            jinghe Jerry He
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: