2012-12-27 15:47:01,191 DEBUG org.apache.hadoop.mapred.Child: Child starting 2012-12-27 15:47:01,475 DEBUG org.apache.hadoop.metrics2.lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of successful kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops) 2012-12-27 15:47:01,477 DEBUG org.apache.hadoop.metrics2.lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of failed kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops) 2012-12-27 15:47:01,483 DEBUG org.apache.hadoop.metrics2.impl.MetricsSystemImpl: UgiMetrics, User and group related metrics 2012-12-27 15:47:01,600 DEBUG org.apache.hadoop.security.Groups: Creating new Groups object 2012-12-27 15:47:01,610 DEBUG org.apache.hadoop.security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000 2012-12-27 15:47:01,629 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login 2012-12-27 15:47:01,629 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login commit 2012-12-27 15:47:01,630 DEBUG org.apache.hadoop.security.UserGroupInformation: using kerberos user:null 2012-12-27 15:47:01,631 DEBUG org.apache.hadoop.security.UserGroupInformation: using local user:UnixPrincipal: kinley 2012-12-27 15:47:01,924 DEBUG org.apache.hadoop.security.UserGroupInformation: UGI loginUser:kinley (auth:KERBEROS) 2012-12-27 15:47:01,925 DEBUG org.apache.hadoop.mapreduce.security.TokenCache: Task: Loaded jobTokenFile from: /mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/jobToken; num of sec keys = 0 Number of tokens 4 2012-12-27 15:47:01,925 DEBUG org.apache.hadoop.mapred.Child: loading token. # keys =0; from file=/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/jobToken 2012-12-27 15:47:01,931 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:job_201212271546_0001 (auth:SIMPLE) from:org.apache.hadoop.mapred.Child.main(Child.java:126) 2012-12-27 15:47:02,093 DEBUG org.apache.hadoop.ipc.Server: rpcKind=RPC_WRITABLE, rpcRequestWrapperClass=class org.apache.hadoop.ipc.WritableRpcEngine$Invocation, rpcInvoker=org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker@3747c1db 2012-12-27 15:47:02,310 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2012-12-27 15:47:02,346 DEBUG org.apache.hadoop.ipc.Client: Use DIGEST authentication for protocol TaskUmbilicalProtocol 2012-12-27 15:47:02,346 DEBUG org.apache.hadoop.ipc.Client: Connecting to /127.0.0.1:59747 2012-12-27 15:47:02,388 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:job_201212271546_0001 (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:585) 2012-12-27 15:47:02,390 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at 127.0.0.1:59747 2012-12-27 15:47:02,556 DEBUG org.apache.hadoop.security.SaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:02,557 DEBUG org.apache.hadoop.security.SaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:02,558 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting username: FWpvYl8yMDEyMTIyNzE1NDZfMDAwMQ== 2012-12-27 15:47:02,558 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:02,558 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:02,559 DEBUG org.apache.hadoop.security.SaslRpcClient: Will send token of size 274 from initSASLContext. 2012-12-27 15:47:02,560 DEBUG org.apache.hadoop.security.SaslRpcClient: Will read input token of size 40 for processing by initSASLContext 2012-12-27 15:47:02,560 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client context established. Negotiated QoP: auth 2012-12-27 15:47:02,564 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #0 2012-12-27 15:47:02,571 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001: starting, having connections 1 2012-12-27 15:47:02,579 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #0 2012-12-27 15:47:02,592 DEBUG org.apache.hadoop.mapred.SortedRanges: currentIndex 0 0:0 2012-12-27 15:47:02,688 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2012-12-27 15:47:02,691 DEBUG org.apache.hadoop.mapred.SortedRanges: currentIndex 0 0:0 2012-12-27 15:47:02,691 DEBUG org.apache.hadoop.mapred.SortedRanges: currentIndex 1 0:0 2012-12-27 15:47:02,699 DEBUG org.apache.hadoop.ipc.RPC: Call: getTask 401 2012-12-27 15:47:02,895 DEBUG org.apache.hadoop.mapred.TaskRunner: mapred.local.dir for child : /mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0 2012-12-27 15:47:02,937 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop library... 2012-12-27 15:47:02,937 DEBUG org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2012-12-27 15:47:03,032 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for all properties in config... 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.auth.token.max.lifetime 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.data.dir 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.checkpoint.txns 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3.replication 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.local.dir 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.disk.healthChecker.interval 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.failed.volumes.tolerated 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.filter.initializers 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.tracker.http.address 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.port 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.userlog.retain.hours 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.leaderport 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.local.dir.minspacestart 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.sort.spill.percent 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.https.server.keystore.resource 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.kerberos.keytab 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.classpath.files 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.parallel.copies 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.hlog.reader.impl 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.id 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.hlog.writer.impl 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.nbreservationblocks 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.file.buffer.size 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.checkpoint.period 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.submit.replication 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.local.dir.minspacekill 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.kill.max 2012-12-27 15:47:03,035 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.profile 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.heartbeats.in.second 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for user.name 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.property.clientPort 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.output.compress 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3.stream-buffer-size 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.healthChecker.interval 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.timeout 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.secondary.http-address 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.temp.dir 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jar 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.decommission.interval 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.http-address 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.info.bindAddress 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.encrypt.data.transfer 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.address 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.is.map 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for jobclient.completion.poll.interval 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.token.validity 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.group 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.queue.default.state 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.failover.max.attempts 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.cache.files.visibilities 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for kfs.client-write-packet-size 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.persist.jobstatus.active 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.mapreduce.scan 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.failover.connection.retries.on.timeouts 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.output.compression.codec 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.global.memstore.lowerLimit 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for file.stream-buffer-size 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.webhdfs.enabled 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.connection.maxidletime 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.ipc.address 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.mapreduce.inputtable 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.policy.file 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.shuffle.merge.percent 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cache.files.filesizes 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.map.max.attempts 2012-12-27 15:47:03,036 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.safemode.extension 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.quorum 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ha.zookeeper.parent-znode 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.memstore.flush.size 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.skip.checksum.errors 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.reduce.input.buffer.percent 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.kerberos.principal 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.dns.interface 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.s3.maxRetries 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.cache.levels 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.sort.factor 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.instrumentation 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.client.scanner.caching 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.userlog.limit.kb 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.kerberos.kinit.command 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.jobTokenFile 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.name.dir.restore 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.rootdir 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.class 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.mapred.output.quorum 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.info.port 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.use.datanode.hostname 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.msginterval 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.util.hash.type 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.seqfile.lazydecompress 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.dns.interface 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.reduce.memory.mb 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for zookeeper.znode.rootserver 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.skip.map.max.skip.records 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.server.versionfile.writeattempts 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.jobhistory.lru.cache.size 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.persist.jobstatus.hours 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for zookeeper.session.timeout 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.group.name 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for local.cache.size 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.df.interval 2012-12-27 15:47:03,037 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.handler.count 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.kerberos.internal.spnego.principal 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.journalnode.rpc-address 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.acl-view-job 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hstore.compaction.max 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.policy 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.replication.interval 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cache.files 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.num.checkpoints.retained 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for zookeeper.znode.parent 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.directoryscan.threads 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping.ldap.ssl 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.server.thread.wakefrequency 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.heartbeat.interval 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for net.topology.script.number.args 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client-write-packet-size 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for sun.security.krb5.debug 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.native.lib.available 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.failover.connection.retries 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.storefile.bloom.cacheonwrite 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.blocksize 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.reduce.shuffle.maxfetchfailures 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.tasktracker.kerberos.principal 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.blockreport.initialDelay 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.sort.mb 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.logcleaner.plugins 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.ha.tail-edits.period 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for file.bytes-per-checksum 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.replication.max 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.map.index.skip 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.profile.maps 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.coprocessor.abortonerror 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tip.id 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.map.tasks.speculative.execution 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.jobtracker.kerberos.principal 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.du.reserved 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.support.append 2012-12-27 15:47:03,038 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.rpc.engine 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.tasktracker.keytab.file 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ftp.blocksize 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.file-block-storage-locations.num-threads 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.server.listen.queue.size 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.hostname.verifier 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.tasks 2012-12-27 15:47:03,047 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.member 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.tasktracker.outofband.heartbeat 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.mapreduce.hfileoutputformat.blocksize 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.logroll.errors.tolerated 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.tmp.dir 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.dns.nameserver 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.delegation.token.renew-interval 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.min.split.size 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.keystores.factory.class 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.client.retries.number 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.sync.behind.writes 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.tasktracker.cache.local.numberdirectories 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.handler.count 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.require.client.cert 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ftp.client-write-packet-size 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.server.tcpnodelay 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.balancer.period 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.rest.readonly 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.fuse.connection.timeout 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.permissions.superusergroup 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.ftp.host.port 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.data.dir.perm 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3native.blocksize 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.dns.nameserver 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.global.memstore.upperLimit 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.replication.min 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.taskmemorymanager.monitoring-interval 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.majorcompaction 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.expiry.interval 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for map.sort.class 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.trash.checkpoint.interval 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.queue.default.acl-administer-jobs 2012-12-27 15:47:03,048 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.cluster.distributed 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.max.tracker.failures 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.name.dir 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.AbstractFileSystem.file.impl 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.handler.count 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.output.value.class 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.jobtracker.split.metainfo.maxsize 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.image.compression.codec 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.online.schema.update.enable 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hash.type 2012-12-27 15:47:03,051 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.complete.cancel.delegation.tokens 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.persist.jobstatus.dir 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.max.filesize 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.sort.record.percent 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for job.end.retry.attempts 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.auth_to_local 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for kfs.bytes-per-checksum 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.partition 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.backup.http-address 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.rpc.protection 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.tasks.speculative.execution 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.https-address 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ftp.stream-buffer-size 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.property.syncLimit 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.ha.log-roll.period 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for file.client-write-packet-size 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.logroll.period 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.tracker.task-controller 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.drop.cache.behind.reads 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.maxtasks.per.job 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.image.transfer.bandwidthPerSec 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hfile.format.version 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.shuffle.ssl.address 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.instrumentation.cmon.jettyport 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.dns.interface 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.mapfile.bloom.size 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.child.log.level 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.max.attempts 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.ha.fencing.ssh.connect-timeout 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3.bytes-per-checksum 2012-12-27 15:47:03,052 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.automatic.close 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cluster.max.map.memory.mb 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.trash.interval 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.authentication 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.defaultFS 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regions.slop 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.server.conf 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.connect.max.retries 2012-12-27 15:47:03,056 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.journalnode.http-address 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.map.output.compression.codec 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for net.topology.node.switch.mapping.impl 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.replication.considerLoad 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.shuffle.input.buffer.percent 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.counters.max 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.user.jobconf.limit 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hfile.index.block.max.size 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.block.write.retries 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.optionallogflushinterval 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.map.index.interval 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.child.java.opts 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.submithost 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.https.keystore.resource 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hstore.blockingStoreFiles 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hfile.block.cache.size 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.mapfile.bloom.error.rate 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.property.initLimit 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.tcpnodelay 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.mapper.new-api 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.relaxed.worker.version.check 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.dir 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.logcleaner.ttl 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.dns.nameserver 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.regionSplitLimit 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.accesstime.precision 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.tracker.report.address 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.instrumentation.cmon.jettyhost 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.seqfile.compress.blocksize 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for keep.failed.task.files 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hstore.compactionThreshold 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.groups.cache.secs 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.cluster.id 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.reduce.input.limit 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cluster.reduce.memory.mb 2012-12-27 15:47:03,057 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for tasktracker.http.threads 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.info.bindAddress 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.authorization 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.ftp.host 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.ifile.readahead 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.memstore.mslab.enabled 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ha.zookeeper.session-timeout.ms 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.https.enable 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.acls.enabled 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.reduce.shuffle.read.timeout 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for zookeeper.znode.acl.parent 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.dns.nameserver 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.blockreport.intervalMsec 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.kerberos.principal 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.balance.bandwidthPerSec 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.max.tracker.blacklists 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for file.blocksize 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.indexcache.mb 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.jobtracker.staging.root.dir 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.security.authentication 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.connect.max.retries.on.timeouts 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ha.zookeeper.acl 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.skip.attempts.to.start.skipping 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.reduce.shuffle.connect.timeout 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.block.access.key.update.interval 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.map.class 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.block.access.token.lifetime 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.reduce.tasks.maximum 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.name 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.preclose.flush.size 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for jobclient.output.filter 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.kerberos.principal 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.restart.recover 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.local.dir 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.checkpoint.check.period 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.enable 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker 2012-12-27 15:47:03,058 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.directoryscan.interval 2012-12-27 15:47:03,059 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.kerberos.principal 2012-12-27 15:47:03,059 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.default.chunk.view.size 2012-12-27 15:47:03,059 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for jobclient.progress.monitor.poll.interval 2012-12-27 15:47:03,059 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.instrumentation.requires.admin 2012-12-27 15:47:03,062 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.safemode.min.datanodes 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.signature.secret.file 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.map.child.log.level 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.https.address 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3native.replication 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.inputformat.class 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.output.compression.type 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for file.replication 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.offheapcache.percentage 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.hdfs.configuration.version 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.retiredjobs.cache.size 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.port 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ipc.client.idlethreshold 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.tmp.dir 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.rs.cacheblocksonwrite 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.decommission.nodes.per.interval 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.skip.worker.version.check 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.peerport 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.defaults.for.version 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.delegation.key.update-interval 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.s3.buffer.dir 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.property.maxClientCnxns 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.dns.interface 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.support.allow.format 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.dns.interface 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.task.profile.reduces 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cluster.map.memory.mb 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid 2012-12-27 15:47:03,063 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.skip.reduce.auto.incr.proc.count 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.ha.automatic-failover.enabled 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.logging.level 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.checkpoint.edits.dir 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.rpc.socket.factory.class.default 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hfile.block.index.cacheonwrite 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.dns.nameserver 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.bulkload.retries.number 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.http.address 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.edits.dir 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for job.end.retry.interval 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.mapred.outputtable 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.fuse.timer.period 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.master.info.port 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for job.local.dir 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.AbstractFileSystem.viewfs.impl 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.queue.names 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3native.stream-buffer-size 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.secondary.namenode.kerberos.internal.spnego.principal 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.job.history.block.size 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.safemode.threshold-pct 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.ifile.readahead.bytes 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.outputformat.class 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.child.tmp 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3native.bytes-per-checksum 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cluster.max.reduce.memory.mb 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for kfs.replication 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.line.input.format.linespermap 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.map.tasks 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.authentication.type 2012-12-27 15:47:03,064 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.failover.sleep.base.millis 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.client.write.buffer 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.info.port.auto 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.submithostaddress 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.map.tasks.maximum 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.jetty.logs.serve.aliases 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.max.transfer.threads 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.auth.key.update.interval 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.handler.count 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.client.conf 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.ssl.enabled 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.AbstractFileSystem.hdfs.impl 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.zookeeper.property.dataDir 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.queue.name 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.client.genericoptionsparser.used 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.regionserver.lease.period 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for kfs.stream-buffer-size 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.defaults.for.version.skip 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.invalidate.work.pct.per.iteration 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.bytes-per-checksum 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.replication 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.permissions.enabled 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.taskScheduler 2012-12-27 15:47:03,065 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.use.datanode.hostname 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.fs-limits.max-directory-items 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3.client-write-packet-size 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.skip.reduce.max.skip.groups 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.failover.sleep.max.millis 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.client.keyvalue.maxsize 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.fs-limits.max-component-length 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3.blocksize 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for tmpjars 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.client.pause 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.edits.journal-plugin.qjournal 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for kfs.blocksize 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.tracker.http.address 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.https.need-auth 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cache.files.timestamps 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.healthChecker.script.timeout 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ftp.replication 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.s3n.block.size 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.instrumentation 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hregion.memstore.block.multiplier 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.num.extra.edits.retained 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.http.staticuser.user 2012-12-27 15:47:03,070 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.output.key.class 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.hstore.blockingWaitTime 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.tasktracker.group 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.system.dir 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.reuse.jvm.num.tasks 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.checkpoint.dir 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for tfile.fs.input.buffer.size 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.skip.map.auto.incr.proc.count 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for tfile.io.chunk.size 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.s3.block.size 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.serializations 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.inmem.merge.threshold 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hbase.rest.port 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.map.memory.mb 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.storefile.bloom.block.size 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.cache.localFiles 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.job.id 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.backup.address 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.seqfile.sorter.recordlimit 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.block.access.token.enable 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.reduce.slowstart.completed.maps 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for s3native.client-write-packet-size 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for ftp.bytes-per-checksum 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.security.group.mapping 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.client.file-block-storage-locations.timeout 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.file.impl.disable.cache 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.tasktracker.tasks.sleeptime-before-sigkill 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.max.objects 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.jobtracker.completeuserjobs.maximum 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.delegation.token.max-lifetime 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.compress.map.output 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.hdfs-blocks-metadata.enabled 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.readahead.bytes 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.image.compress 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.shuffle.ssl.enabled 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.merge.recordsBeforeProgress 2012-12-27 15:47:03,071 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.stream-buffer-size 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for tfile.fs.output.buffer.size 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.permissions.umask-mode 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.datanode.drop.cache.behind.writes 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for hadoop.common.configuration.version 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for dfs.namenode.replication.work.multiplier.per.iteration 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.job.acl-modify-job 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for io.seqfile.local.dir 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapreduce.shuffle.ssl.port 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for fs.s3.sleepTimeSeconds 2012-12-27 15:47:03,072 DEBUG org.apache.hadoop.conf.Configuration: Handling deprecation for mapred.working.dir 2012-12-27 15:47:03,380 DEBUG org.apache.hadoop.mapred.TaskRunner: Fully deleting contents of /mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/work 2012-12-27 15:47:03,445 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2012-12-27 15:47:03,449 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2012-12-27 15:47:03,461 DEBUG org.apache.hadoop.mapred.Child: Creating remote user to execute task: kinley 2012-12-27 15:47:03,461 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:03,499 DEBUG org.apache.hadoop.hdfs.NameNodeProxies: multipleLinearRandomRetry = null 2012-12-27 15:47:03,504 DEBUG org.apache.hadoop.ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWritable, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@4e46b90a 2012-12-27 15:47:03,696 DEBUG org.apache.hadoop.hdfs.DFSClient: Short circuit read is false 2012-12-27 15:47:03,713 DEBUG org.apache.hadoop.mapred.Task: using new api for output committer 2012-12-27 15:47:03,860 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection opening connection to ZooKeeper with ensemble (three.kinley.com:2181) 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-cdh4.1.2--1, built on 11/01/2012 16:46 GMT 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=one.kinley.com 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_31 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/run/cloudera-scm-agent/process/810-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-mapreduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.1.1.jar:/usr/share/cmf/lib/plugins/event-publish-4.1.1-shaded.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/hue-plugins-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/hadoop-auth.jar:/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/jars/classes:/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/jars/job.jar:/mapred/local/taskTracker/kinley/distcache/-906880726833934237_1895045514_1561026234/one.kinley.com/user/kinley/.staging/job_201212271546_0001/libjars/hbase-0.92.1-cdh4.1.2-security.jar:/mapred/local/taskTracker/kinley/distcache/5118981467625389877_-1206898784_1561026314/one.kinley.com/user/kinley/.staging/job_201212271546_0001/libjars/zookeeper-3.4.3-cdh4.1.2.jar:/mapred/local/taskTracker/kinley/distcache/-6566020904240930122_-853302606_1561026419/one.kinley.com/user/kinley/.staging/job_201212271546_0001/libjars/hadoop-core-2.0.0-mr1-cdh4.1.2.jar:/mapred/local/taskTracker/kinley/distcache/1202549491262581484_-657330209_1561026517/one.kinley.com/user/kinley/.staging/job_201212271546_0001/libjars/hadoop-common-2.0.0-cdh4.1.2.jar:/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/work 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hadoop-0.20-mapreduce/lib/native/Linux-amd64-64:/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/work 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/work/tmp 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler= 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.2.0-23-generic 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=kinley 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/kinley 2012-12-27 15:47:03,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/mapred/local/taskTracker/kinley/jobcache/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/work 2012-12-27 15:47:03,869 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=three.kinley.com:2181 sessionTimeout=60000 watcher=hconnection 2012-12-27 15:47:03,871 DEBUG org.apache.zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false 2012-12-27 15:47:03,893 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server three.kinley.com/172.16.223.6:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration) 2012-12-27 15:47:03,894 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 14485@one.kinley.com 2012-12-27 15:47:03,900 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to three.kinley.com/172.16.223.6:2181, initiating session 2012-12-27 15:47:03,901 DEBUG org.apache.zookeeper.ClientCnxn: Session establishment request sent on three.kinley.com/172.16.223.6:2181 2012-12-27 15:47:03,906 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server three.kinley.com/172.16.223.6:2181, sessionid = 0x13bdbb8579f005d, negotiated timeout = 40000 2012-12-27 15:47:03,907 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2012-12-27 15:47:03,910 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection-0x13bdbb8579f005d connected 2012-12-27 15:47:03,912 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 1,3 replyHeader:: 1,3979,0 request:: '/hbase/master,T response:: s{3923,3923,1356623020641,1356623020641,0,0,0,88906202454360147,67,0,3923} 2012-12-27 15:47:03,912 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Set watcher on existing znode /hbase/master 2012-12-27 15:47:03,914 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,3979,0 request:: '/hbase/master,T response:: #ffffffff00017313735363940696d70616c612e6b696e6c65792e636f6d00696d70616c612e6b696e6c65792e636f6d2c36303030302c31333536363233303139323932,s{3923,3923,1356623020641,1356623020641,0,0,0,88906202454360147,67,0,3923} 2012-12-27 15:47:03,914 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Retrieved 39 byte(s) of data from znode /hbase/master and set watcher; \x00\x00three.kinley.com,600... 2012-12-27 15:47:03,916 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 3,3 replyHeader:: 3,3979,0 request:: '/hbase/root-region-server,T response:: s{3942,3942,1356623030193,1356623030193,0,0,0,0,65,0,3942} 2012-12-27 15:47:03,916 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Set watcher on existing znode /hbase/root-region-server 2012-12-27 15:47:03,917 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 4,4 replyHeader:: 4,3979,0 request:: '/hbase/root-region-server,T response:: #ffffffff00017313737333640696d70616c612e6b696e6c65792e636f6d696d70616c612e6b696e6c65792e636f6d2c36303032302c31333536363233303235313731,s{3942,3942,1356623030193,1356623030193,0,0,0,0,65,0,3942} 2012-12-27 15:47:03,917 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Retrieved 37 byte(s) of data from znode /hbase/root-region-server and set watcher; three.kinley.com,60020,13566... 2012-12-27 15:47:03,924 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 5,3 replyHeader:: 5,3979,0 request:: '/hbase,F response:: s{5,5,1353337471371,1353337471371,0,160,0,0,0,12,3942} 2012-12-27 15:47:03,926 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 6,4 replyHeader:: 6,3979,0 request:: '/hbase/root-region-server,T response:: #ffffffff00017313737333640696d70616c612e6b696e6c65792e636f6d696d70616c612e6b696e6c65792e636f6d2c36303032302c31333536363233303235313731,s{3942,3942,1356623030193,1356623030193,0,0,0,0,65,0,3942} 2012-12-27 15:47:03,926 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Retrieved 37 byte(s) of data from znode /hbase/root-region-server and set watcher; three.kinley.com,60020,13566... 2012-12-27 15:47:03,926 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@35f38fc6; serverName=three.kinley.com,60020,1356623025171 2012-12-27 15:47:03,928 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 7,3 replyHeader:: 7,3979,0 request:: '/hbase/hbaseid,F response:: s{50,3925,1353338553046,1356623021747,37,0,0,0,64,0,50} 2012-12-27 15:47:03,929 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb8579f005d, packet:: clientPath:null serverPath:null finished:false header:: 8,4 replyHeader:: 8,3979,0 request:: '/hbase/hbaseid,F response:: #ffffffff00017313735363940696d70616c612e6b696e6c65792e636f6d64343635336133372d383364662d343932392d623935302d653033393837633362346538,s{50,3925,1353338553046,1356623021747,37,0,0,0,64,0,50} 2012-12-27 15:47:03,929 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb8579f005d Retrieved 36 byte(s) of data from znode /hbase/hbaseid; data=d4653a37-83df-4929-b950-e0398... 2012-12-27 15:47:03,933 DEBUG org.apache.hadoop.ipc.HBaseRPC: Using org.apache.hadoop.hbase.ipc.SecureRpcEngine for org.apache.hadoop.hbase.ipc.HRegionInterface 2012-12-27 15:47:03,951 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:03,956 DEBUG org.apache.hadoop.ipc.HBaseClient: The ping interval is60000ms. 2012-12-27 15:47:03,957 DEBUG org.apache.hadoop.ipc.HBaseClient: >>> HBaseClient: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:03,958 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> caching SecureClient: 918201446 2012-12-27 15:47:03,963 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:03,963 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:03,963 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:03,965 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:03,965 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:03,965 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:03,965 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:03,966 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:03,966 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:03,966 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:03,966 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:03,966 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:03,972 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/three.kinley.com@KINLEY.COM 2012-12-27 15:47:03,973 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:03,974 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:03,974 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to three.kinley.com/172.16.223.6:60020 2012-12-27 15:47:03,987 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:03,989 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:03,990 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:03,992 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:03,993 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:03,993 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:03,993 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:03,993 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:03,993 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:03,996 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:03,996 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 40 for processing by initSASLContext 2012-12-27 15:47:03,996 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client context established. Negotiated QoP: auth 2012-12-27 15:47:03,997 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #0 2012-12-27 15:47:04,010 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley: starting, having connections 1 2012-12-27 15:47:04,024 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #0 2012-12-27 15:47:04,024 DEBUG org.apache.hadoop.ipc.SecureClient: call #0 state is 0 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: call #0, response is: OW[class=long,value=29] 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: getProtocolVersion 63 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,025 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,025 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,025 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,026 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,026 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,026 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,026 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,026 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,026 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #1 2012-12-27 15:47:04,027 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #1 2012-12-27 15:47:04,027 DEBUG org.apache.hadoop.ipc.SecureClient: call #1 state is 0 2012-12-27 15:47:04,027 DEBUG org.apache.hadoop.ipc.SecureClient: call #1, response is: OW[class=long,value=29] 2012-12-27 15:47:04,027 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: getProtocolVersion 2 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,028 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,028 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,028 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,028 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,028 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,029 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #2 2012-12-27 15:47:04,031 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #2 2012-12-27 15:47:04,031 DEBUG org.apache.hadoop.ipc.SecureClient: call #2 state is 0 2012-12-27 15:47:04,038 DEBUG org.apache.hadoop.ipc.SecureClient: call #2, response is: OW[class=class org.apache.hadoop.hbase.client.Result,value=keyvalues={.META.,,1/info:regioninfo/1353338552907/Put/vlen=34, .META.,,1/info:server/1356623031451/Put/vlen=23, .META.,,1/info:serverstartcode/1356623031451/Put/vlen=8, .META.,,1/info:v/1353338552907/Put/vlen=2}] 2012-12-27 15:47:04,038 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: getClosestRowBefore 11 2012-12-27 15:47:04,041 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Cached location for .META.,,1.1028785192 is three.kinley.com:60020 2012-12-27 15:47:04,064 WARN org.apache.hadoop.conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available 2012-12-27 15:47:04,121 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,121 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,121 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,122 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,122 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,122 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,122 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,122 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #3 2012-12-27 15:47:04,124 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #3 2012-12-27 15:47:04,124 DEBUG org.apache.hadoop.ipc.SecureClient: call #3 state is 0 2012-12-27 15:47:04,125 DEBUG org.apache.hadoop.ipc.SecureClient: call #3, response is: OW[class=class org.apache.hadoop.hbase.client.Result,value=keyvalues={Converged-headers,,1352794812069.a8c9d392f1958c3328118594b236aba2./info:regioninfo/1355830338184/Put/vlen=102, Converged-headers,,1352794812069.a8c9d392f1958c3328118594b236aba2./info:server/1356623031730/Put/vlen=23, Converged-headers,,1352794812069.a8c9d392f1958c3328118594b236aba2./info:serverstartcode/1356623031730/Put/vlen=8}] 2012-12-27 15:47:04,125 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: getClosestRowBefore 4 2012-12-27 15:47:04,126 DEBUG org.apache.hadoop.hbase.client.MetaScanner: Scanning .META. starting at row=Converged-headers,,00000000000000 for max=10 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@35f38fc6 2012-12-27 15:47:04,126 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,126 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,126 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,126 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,129 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,129 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,129 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,129 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,129 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,129 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,129 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,129 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,129 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #4 2012-12-27 15:47:04,131 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #4 2012-12-27 15:47:04,131 DEBUG org.apache.hadoop.ipc.SecureClient: call #4 state is 0 2012-12-27 15:47:04,131 DEBUG org.apache.hadoop.ipc.SecureClient: call #4, response is: OW[class=long,value=4266639818219147991] 2012-12-27 15:47:04,131 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: openScanner 5 2012-12-27 15:47:04,131 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,132 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,132 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,132 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,132 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,132 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #5 2012-12-27 15:47:04,134 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #5 2012-12-27 15:47:04,134 DEBUG org.apache.hadoop.ipc.SecureClient: call #5 state is 0 2012-12-27 15:47:04,134 DEBUG org.apache.hadoop.ipc.SecureClient: call #5, response is: OW[class=class [Lorg.apache.hadoop.hbase.client.Result;,value=[Lorg.apache.hadoop.hbase.client.Result;@5e36d047] 2012-12-27 15:47:04,134 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: next 3 2012-12-27 15:47:04,134 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Cached location for Converged-headers,,1352794812069.a8c9d392f1958c3328118594b236aba2. is three.kinley.com:60020 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId three.kinley.com/172.16.223.6:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,135 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,135 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,135 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,135 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,135 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley sending #6 2012-12-27 15:47:04,136 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley got value #6 2012-12-27 15:47:04,136 DEBUG org.apache.hadoop.ipc.SecureClient: call #6 state is 0 2012-12-27 15:47:04,137 DEBUG org.apache.hadoop.ipc.SecureClient: call #6, response is: OW[class=void,value=null] 2012-12-27 15:47:04,138 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: Call: close 4 2012-12-27 15:47:04,138 INFO org.apache.hadoop.hbase.mapreduce.TableOutputFormat: Created table instance for Converged-headers 2012-12-27 15:47:04,146 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2012-12-27 15:47:04,149 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7b48392 2012-12-27 15:47:04,294 DEBUG org.apache.hadoop.util.ProcfsBasedProcessTree: [ 14485 ] 2012-12-27 15:47:04,298 DEBUG org.apache.hadoop.hbase.mapreduce.TableInputFormat: hbase.cluster.id: 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:04,298 DEBUG org.apache.hadoop.hbase.mapreduce.TableInputFormat: hbase.zookeeper.quorum: two.kinley.com 2012-12-27 15:47:04,305 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection opening connection to ZooKeeper with ensemble (two.kinley.com:2181) 2012-12-27 15:47:04,305 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=two.kinley.com:2181 sessionTimeout=60000 watcher=hconnection 2012-12-27 15:47:04,313 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server two.kinley.com/172.16.223.4:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration) 2012-12-27 15:47:04,313 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 14485@one.kinley.com 2012-12-27 15:47:04,313 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to two.kinley.com/172.16.223.4:2181, initiating session 2012-12-27 15:47:04,314 DEBUG org.apache.zookeeper.ClientCnxn: Session establishment request sent on two.kinley.com/172.16.223.4:2181 2012-12-27 15:47:04,316 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server two.kinley.com/172.16.223.4:2181, sessionid = 0x13bdbb3322f0090, negotiated timeout = 60000 2012-12-27 15:47:04,316 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2012-12-27 15:47:04,318 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection-0x13bdbb3322f0090 connected 2012-12-27 15:47:04,321 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 1,3 replyHeader:: 1,100620,0 request:: '/hbase/master,T response:: s{100568,100568,1356623056582,1356623056582,0,0,0,88906180351426694,65,0,100568} 2012-12-27 15:47:04,321 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Set watcher on existing znode /hbase/master 2012-12-27 15:47:04,323 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,100620,0 request:: '/hbase/master,T response:: #ffffffff0001631393630304068626173652e6b696e6c65792e636f6d0068626173652e6b696e6c65792e636f6d2c36303030302c31333536363233303533363638,s{100568,100568,1356623056582,1356623056582,0,0,0,88906180351426694,65,0,100568} 2012-12-27 15:47:04,323 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 38 byte(s) of data from znode /hbase/master and set watcher; \x00\x00two.kinley.com,6000... 2012-12-27 15:47:04,324 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 3,3 replyHeader:: 3,100620,0 request:: '/hbase/root-region-server,T response:: s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:04,324 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Set watcher on existing znode /hbase/root-region-server 2012-12-27 15:47:04,325 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 4,4 replyHeader:: 4,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:04,325 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:04,326 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 5,3 replyHeader:: 5,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:04,327 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 6,4 replyHeader:: 6,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:04,327 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:04,327 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:04,328 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 7,3 replyHeader:: 7,100620,0 request:: '/hbase/hbaseid,F response:: s{12,100570,1343652490051,1356623058001,114,0,0,0,63,0,12} 2012-12-27 15:47:04,328 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 8,4 replyHeader:: 8,100620,0 request:: '/hbase/hbaseid,F response:: #ffffffff0001631393630304068626173652e6b696e6c65792e636f6d37626230316338342d336336362d343963612d383264322d326639323138663133646164,s{12,100570,1343652490051,1356623058001,114,0,0,0,63,0,12} 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/hbaseid; data=7bb01c84-3c66-49ca-82d2-2f921... 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:04,329 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,329 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:04,329 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:04,330 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,330 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:04,330 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:04,331 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:04,331 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:04,331 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:04,332 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:04,332 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:04,333 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:04,333 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:04,333 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:04,333 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:04,335 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:04,336 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:04,337 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:04,338 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:04,338 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:04,339 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:04,343 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:04,343 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:04,344 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=0 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:04,345 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 9,3 replyHeader:: 9,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:04,346 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 10,4 replyHeader:: 10,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:04,346 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:04,346 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:05,348 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 11,3 replyHeader:: 11,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:05,349 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 12,4 replyHeader:: 12,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:05,349 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:05,350 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:05,350 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:05,350 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:05,350 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:05,350 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:05,351 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:05,352 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:05,352 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:05,353 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:05,355 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:05,356 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:05,356 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:05,356 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:05,356 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:05,356 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:05,356 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:05,356 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:05,357 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=1 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:05,358 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 13,3 replyHeader:: 13,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:05,365 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 14,4 replyHeader:: 14,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:05,365 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:05,365 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:06,368 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 15,3 replyHeader:: 15,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:06,369 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 16,4 replyHeader:: 16,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:06,369 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:06,370 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:06,370 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:06,370 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:06,370 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:06,370 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:06,371 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:06,371 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:06,371 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:06,372 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:06,372 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:06,372 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:06,372 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:06,372 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:06,373 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:06,382 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:06,382 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:06,382 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:06,383 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:06,383 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:06,383 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:06,383 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:06,383 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:06,385 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=2 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:06,386 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 17,3 replyHeader:: 17,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:06,390 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 18,4 replyHeader:: 18,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:06,390 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:06,391 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:06,720 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #1 2012-12-27 15:47:06,721 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #1 2012-12-27 15:47:06,721 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:07,393 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 19,3 replyHeader:: 19,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:07,393 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 20,4 replyHeader:: 20,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:07,394 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:07,394 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:07,394 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:07,395 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:07,395 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:07,395 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:07,396 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:07,396 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:07,396 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:07,397 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:07,397 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:07,397 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:07,397 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:07,397 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:07,398 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:07,400 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:07,400 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:07,400 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:07,400 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:07,400 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:07,400 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:07,401 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:07,401 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:07,401 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=3 of 10 failed; retrying after sleep of 2000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:07,402 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 21,3 replyHeader:: 21,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:07,403 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 22,4 replyHeader:: 22,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:07,403 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:07,403 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:09,405 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 23,3 replyHeader:: 23,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:09,406 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 24,4 replyHeader:: 24,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:09,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:09,406 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:09,407 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:09,407 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:09,407 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:09,407 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:09,407 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:09,408 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:09,414 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:09,415 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:09,422 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:09,422 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:09,422 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:09,423 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:09,423 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:09,423 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:09,424 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:09,424 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:09,424 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=4 of 10 failed; retrying after sleep of 2000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:09,425 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 25,3 replyHeader:: 25,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:09,427 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 26,4 replyHeader:: 26,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:09,427 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:09,427 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:09,721 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #2 2012-12-27 15:47:09,723 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #2 2012-12-27 15:47:09,723 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:11,429 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 27,3 replyHeader:: 27,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:11,430 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 28,4 replyHeader:: 28,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:11,430 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:11,430 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:11,431 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:11,431 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:11,431 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:11,431 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:11,432 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:11,432 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:11,432 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:11,433 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:11,436 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:11,436 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:11,436 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:11,436 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:11,436 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:11,437 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:11,437 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:11,437 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:11,437 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=5 of 10 failed; retrying after sleep of 4000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:11,438 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 29,3 replyHeader:: 29,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:11,439 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 30,4 replyHeader:: 30,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:11,439 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:11,440 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:12,723 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #3 2012-12-27 15:47:12,725 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #3 2012-12-27 15:47:12,725 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:14,135 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley: closed 2012-12-27 15:47:14,135 DEBUG org.apache.hadoop.ipc.HBaseClient: IPC Client (918201446) connection to three.kinley.com/172.16.223.6:60020 from kinley: stopped, remaining connections 0 2012-12-27 15:47:15,446 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 31,3 replyHeader:: 31,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:15,448 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 32,4 replyHeader:: 32,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:15,448 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:15,449 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:15,449 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:15,449 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:15,449 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:15,449 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:15,450 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:15,450 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:15,450 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:15,450 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:15,451 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:15,452 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:15,454 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:15,454 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:15,454 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:15,455 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:15,455 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:15,455 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:15,455 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:15,455 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:15,456 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=6 of 10 failed; retrying after sleep of 4000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:15,457 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 33,3 replyHeader:: 33,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:15,458 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 34,4 replyHeader:: 34,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:15,458 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:15,458 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:15,726 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #4 2012-12-27 15:47:15,727 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #4 2012-12-27 15:47:15,727 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:47:17,269 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x13bdbb8579f005d after 0ms 2012-12-27 15:47:18,728 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #5 2012-12-27 15:47:18,729 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #5 2012-12-27 15:47:18,729 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:19,459 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 35,3 replyHeader:: 35,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:19,460 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 36,4 replyHeader:: 36,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:19,460 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:19,461 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:19,461 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:19,461 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:19,461 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:19,461 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:19,462 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:19,462 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:19,462 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:19,462 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:19,462 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:19,466 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2012-12-27 15:47:19,466 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:19,466 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:19,467 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:19,467 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:19,467 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:19,467 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:19,467 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:19,468 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:19,471 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:19,471 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:19,471 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:19,472 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:19,472 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:19,472 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:19,472 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:19,472 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:19,472 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=7 of 10 failed; retrying after sleep of 8000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:19,474 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 37,3 replyHeader:: 37,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:19,475 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 38,4 replyHeader:: 38,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:19,475 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:19,475 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:21,730 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #6 2012-12-27 15:47:21,731 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #6 2012-12-27 15:47:21,731 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:24,732 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #7 2012-12-27 15:47:24,733 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #7 2012-12-27 15:47:24,733 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:27,477 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 39,3 replyHeader:: 39,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:27,478 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 40,4 replyHeader:: 40,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:27,478 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:27,479 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:27,479 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:27,479 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:27,479 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:27,479 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:27,480 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:27,480 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:27,480 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:27,481 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:27,481 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:27,481 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:27,481 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:27,481 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:27,482 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:27,483 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:27,484 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:27,484 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:27,484 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:27,484 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:27,484 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:27,484 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:27,484 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:27,485 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=8 of 10 failed; retrying after sleep of 16000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:27,487 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 41,3 replyHeader:: 41,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:27,488 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 42,4 replyHeader:: 42,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:27,488 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:27,488 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:27,734 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #8 2012-12-27 15:47:27,735 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #8 2012-12-27 15:47:27,735 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:30,613 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x13bdbb8579f005d after 1ms 2012-12-27 15:47:30,736 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #9 2012-12-27 15:47:30,737 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #9 2012-12-27 15:47:30,737 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:47:33,739 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #10 2012-12-27 15:47:33,741 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #10 2012-12-27 15:47:33,741 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:36,742 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #11 2012-12-27 15:47:36,743 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #11 2012-12-27 15:47:36,743 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:39,743 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #12 2012-12-27 15:47:39,745 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #12 2012-12-27 15:47:39,746 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 3 2012-12-27 15:47:42,746 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #13 2012-12-27 15:47:42,748 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #13 2012-12-27 15:47:42,748 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:43,491 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 43,3 replyHeader:: 43,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:43,492 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 44,4 replyHeader:: 44,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:43,492 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:43,493 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:43,493 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:43,493 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:43,493 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:43,493 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:43,493 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:43,495 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:43,495 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:43,496 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:43,496 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:43,496 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:43,496 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:43,497 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:43,497 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:43,497 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:43,499 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:43,499 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:43,499 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:43,499 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:43,499 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:43,500 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:43,503 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:43,503 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,503 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:43,504 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,504 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,504 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:43,504 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:43,504 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:915) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:43,505 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=.META., metaLocation=null, attempt=0 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,507 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 45,3 replyHeader:: 45,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:43,508 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 46,4 replyHeader:: 46,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:43,508 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:43,508 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:43,509 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:43,509 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:43,509 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:43,509 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:43,509 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:43,510 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:43,510 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:43,510 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:43,510 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:43,511 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:43,511 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:43,511 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:43,512 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:43,512 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:43,512 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:43,512 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:43,512 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:43,513 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:43,515 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:43,515 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,515 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:43,515 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,515 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,515 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:43,516 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:43,516 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:43,516 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=0 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:43,519 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 47,3 replyHeader:: 47,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:43,521 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 48,4 replyHeader:: 48,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:43,521 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:43,521 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:43,958 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x13bdbb8579f005d after 0ms 2012-12-27 15:47:44,523 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 49,3 replyHeader:: 49,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:44,524 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 50,4 replyHeader:: 50,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:44,524 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:44,524 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:44,525 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:44,525 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:44,525 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:44,525 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:44,525 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:44,525 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:44,526 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:44,526 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:44,526 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:44,526 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:44,531 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:44,531 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:44,532 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:44,532 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:44,532 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:44,533 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:44,533 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:44,533 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:44,533 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:44,533 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:44,534 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:44,536 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:44,536 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:44,536 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:44,536 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:44,536 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:44,536 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:44,537 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:44,537 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:44,537 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=1 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:44,538 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 51,3 replyHeader:: 51,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:44,539 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 52,4 replyHeader:: 52,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:44,539 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:44,539 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:45,541 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 53,3 replyHeader:: 53,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:45,542 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 54,4 replyHeader:: 54,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:45,542 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:45,543 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:45,543 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:45,543 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:45,543 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:45,543 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:45,544 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:45,544 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:45,548 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:45,551 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:45,551 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:45,551 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:45,551 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:45,551 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:45,552 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:45,554 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:45,555 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:45,555 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:45,555 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:45,555 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:45,555 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:45,556 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:45,556 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:45,559 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=2 of 10 failed; retrying after sleep of 1000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:45,565 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 55,3 replyHeader:: 55,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:45,566 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 56,4 replyHeader:: 56,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:45,566 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:45,566 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:45,748 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #14 2012-12-27 15:47:45,750 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #14 2012-12-27 15:47:45,750 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:46,568 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 57,3 replyHeader:: 57,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:46,569 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 58,4 replyHeader:: 58,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:46,569 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:46,570 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:46,570 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:46,571 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:46,571 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:46,571 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:46,571 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:46,572 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:46,572 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:46,572 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:46,573 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:46,574 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:46,574 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:46,574 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:46,574 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:46,574 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:46,583 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:46,583 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:46,583 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:46,584 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:46,584 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:46,584 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:46,584 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:46,584 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:46,584 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=3 of 10 failed; retrying after sleep of 2000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:46,586 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 59,3 replyHeader:: 59,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:46,588 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 60,4 replyHeader:: 60,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:46,588 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:46,588 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:48,590 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 61,3 replyHeader:: 61,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:48,591 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 62,4 replyHeader:: 62,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:48,591 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:48,592 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:48,592 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:48,592 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:48,592 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:48,592 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:48,593 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:48,593 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:48,593 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:48,593 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:48,593 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:48,594 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:48,594 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:48,595 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:48,596 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:48,596 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:48,596 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:48,596 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:48,596 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:48,597 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:48,600 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:48,600 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:48,600 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:48,600 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:48,600 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:48,600 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:48,601 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:48,601 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:48,601 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=4 of 10 failed; retrying after sleep of 2000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:48,602 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 63,3 replyHeader:: 63,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:48,603 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 64,4 replyHeader:: 64,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:48,603 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:48,603 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:48,751 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #15 2012-12-27 15:47:48,752 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #15 2012-12-27 15:47:48,763 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 13 2012-12-27 15:47:50,606 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 65,3 replyHeader:: 65,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:50,607 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 66,4 replyHeader:: 66,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:50,607 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:50,608 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:50,608 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:50,608 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:50,608 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:50,608 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:50,609 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:50,609 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:50,609 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:50,610 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:50,610 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:50,610 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:50,610 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:50,610 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:50,611 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:50,612 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:50,612 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:50,613 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:50,613 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:50,613 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:50,613 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:50,613 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:50,613 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:50,614 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=5 of 10 failed; retrying after sleep of 4000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:50,616 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 67,3 replyHeader:: 67,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:50,617 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 68,4 replyHeader:: 68,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:50,617 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:50,617 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:51,764 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #16 2012-12-27 15:47:51,765 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #16 2012-12-27 15:47:51,766 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:47:54,619 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 69,3 replyHeader:: 69,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:54,620 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 70,4 replyHeader:: 70,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:54,620 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:54,620 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:54,620 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:54,621 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:54,621 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:54,621 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:54,621 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:54,621 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:54,622 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:54,622 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:54,622 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:54,623 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:54,623 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:54,623 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:54,623 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:54,623 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:54,624 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:54,625 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:54,626 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:54,626 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:54,626 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:54,626 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:54,626 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:54,626 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:54,626 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:54,627 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=6 of 10 failed; retrying after sleep of 4000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:54,628 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 71,3 replyHeader:: 71,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:54,629 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 72,4 replyHeader:: 72,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:54,629 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:54,629 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:54,766 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #17 2012-12-27 15:47:54,767 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #17 2012-12-27 15:47:54,767 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:47:57,303 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x13bdbb8579f005d after 0ms 2012-12-27 15:47:57,768 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #18 2012-12-27 15:47:57,769 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #18 2012-12-27 15:47:57,769 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:47:58,631 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 73,3 replyHeader:: 73,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:58,632 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 74,4 replyHeader:: 74,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:47:58,632 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:58,632 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:47:58,633 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:47:58,633 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:58,633 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:47:58,633 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:47:58,634 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:58,634 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:47:58,634 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:47:58,635 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:47:58,635 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:47:58,637 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:47:58,637 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:47:58,637 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:47:58,637 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:47:58,639 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:47:58,639 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:58,639 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:47:58,640 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:58,640 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:58,640 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:58,640 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:47:58,640 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:47:58,641 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=7 of 10 failed; retrying after sleep of 8000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:47:58,642 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 75,3 replyHeader:: 75,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:47:58,643 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 76,4 replyHeader:: 76,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:47:58,643 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:47:58,643 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:48:00,770 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #19 2012-12-27 15:48:00,771 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #19 2012-12-27 15:48:00,771 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:03,771 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #20 2012-12-27 15:48:03,773 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #20 2012-12-27 15:48:03,773 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:06,645 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 77,3 replyHeader:: 77,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:48:06,646 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 78,4 replyHeader:: 78,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:48:06,646 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:48:06,647 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:48:06,647 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:48:06,647 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:48:06,647 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:48:06,647 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:48:06,648 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:48:06,648 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:48:06,653 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:48:06,655 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:48:06,658 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:48:06,658 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:06,658 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:48:06,663 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:06,663 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:06,663 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:06,663 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:48:06,663 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:06,663 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-ROOT-,,0.70236052, hostname=two.kinley.com, port=60020}, attempt=8 of 10 failed; retrying after sleep of 16000 because: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:06,667 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 79,3 replyHeader:: 79,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:48:06,668 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 80,4 replyHeader:: 80,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:48:06,668 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:48:06,668 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:48:06,773 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #21 2012-12-27 15:48:06,775 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #21 2012-12-27 15:48:06,775 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:09,775 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #22 2012-12-27 15:48:09,777 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #22 2012-12-27 15:48:09,777 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:10,644 DEBUG org.apache.zookeeper.ClientCnxn: Got ping response for sessionid: 0x13bdbb8579f005d after 0ms 2012-12-27 15:48:12,778 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #23 2012-12-27 15:48:12,779 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #23 2012-12-27 15:48:12,779 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:48:15,780 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #24 2012-12-27 15:48:15,782 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #24 2012-12-27 15:48:15,782 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:18,782 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #25 2012-12-27 15:48:18,784 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #25 2012-12-27 15:48:18,784 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 2 2012-12-27 15:48:21,784 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #26 2012-12-27 15:48:21,785 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #26 2012-12-27 15:48:21,785 DEBUG org.apache.hadoop.ipc.RPC: Call: ping 1 2012-12-27 15:48:22,670 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 81,3 replyHeader:: 81,100620,0 request:: '/hbase,F response:: s{3,3,1343652486419,1343652486419,0,706,0,0,0,12,100578} 2012-12-27 15:48:22,671 DEBUG org.apache.zookeeper.ClientCnxn: Reading reply sessionid:0x13bdbb3322f0090, packet:: clientPath:null serverPath:null finished:false header:: 82,4 replyHeader:: 82,100620,0 request:: '/hbase/root-region-server,T response:: #ffffffff0001631393539344068626173652e6b696e6c65792e636f6d68626173652e6b696e6c65792e636f6d2c36303032302c31333536363233303533363337,s{100578,100578,1356623063972,1356623063972,0,0,0,0,63,0,100578} 2012-12-27 15:48:22,671 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x13bdbb3322f0090 Retrieved 36 byte(s) of data from znode /hbase/root-region-server and set watcher; two.kinley.com,60020,135662... 2012-12-27 15:48:22,671 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@64d22462; serverName=two.kinley.com,60020,1356623053637 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureRpcEngine: >>> get SecureClient: 918201446 : 7bb01c84-3c66-49ca-82d2-2f9218f13dad 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: >>> current clusterId: d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: >>> remoteId two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE) 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: >>> Users tokens: 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HDFS_DELEGATION_TOKEN, Service: 172.16.223.3:8020, Ident: (HDFS_DELEGATION_TOKEN token 320 for kinley) 2012-12-27 15:48:22,672 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: d4653a37-83df-4929-b950-e03987c3b4e8, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 2d 8a 01 3b dd 0b 56 92 8a 01 3c 01 17 da 92 01 2012-12-27 15:48:22,672 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: HBASE_AUTH_TOKEN, Service: 7bb01c84-3c66-49ca-82d2-2f9218f13dad, Ident: 01 00 00 00 17 6b 69 6e 6c 65 79 2f 61 64 6d 69 6e 40 4b 49 4e 4c 45 59 2e 43 4f 4d 18 8a 01 3b dd 0b 56 f0 8a 01 3c 01 17 da f0 03 2012-12-27 15:48:22,672 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: job_201212271546_0001, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:48:22,672 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind mapreduce.job 2012-12-27 15:48:22,672 DEBUG org.apache.hadoop.ipc.SecureClient: Kind: mapreduce.job, Service: 127.0.0.1:59747, Ident: 15 6a 6f 62 5f 32 30 31 32 31 32 32 37 31 35 34 36 5f 30 30 30 31 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.ipc.SecureClient: RPC Server Kerberos principal name for protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is hbase/two.kinley.com@KINLEY.COM 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.ipc.SecureClient: Use DIGEST authentication for protocol HRegionInterface 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.ipc.SecureClient: >>> caching new connection for [two.kinley.com/172.16.223.4:60020:interface org.apache.hadoop.hbase.ipc.HRegionInterface:kinley (auth:SIMPLE)] 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.ipc.SecureClient: Connecting to two.kinley.com/172.16.223.4:60020 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:48:22,673 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at d4653a37-83df-4929-b950-e03987c3b4e8 2012-12-27 15:48:22,674 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:48:22,674 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 0 2012-12-27 15:48:22,674 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:48:22,675 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting username: AQAAABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTS2KATvdC1aSigE8ARfakgE= 2012-12-27 15:48:22,675 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:48:22,675 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:48:22,675 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: Will send token of size 302 from initSASLContext. 2012-12-27 15:48:22,677 DEBUG org.apache.hadoop.hbase.security.HBaseSaslRpcClient: >>> inStream:-33 : 1 2012-12-27 15:48:22,677 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:22,677 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 2012-12-27 15:48:22,677 WARN org.apache.hadoop.ipc.SecureClient: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:22,677 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) 2012-12-27 15:48:22,677 DEBUG org.apache.hadoop.ipc.SecureClient: closing ipc connection to two.kinley.com/172.16.223.4:60020: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:22,678 DEBUG org.apache.hadoop.ipc.SecureClient: IPC Client (918201446) connection to two.kinley.com/172.16.223.4:60020 from kinley: closed 2012-12-27 15:48:22,678 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:22,678 ERROR org.apache.hadoop.hbase.mapreduce.TableInputFormat: org.apache.hadoop.security.token.SecretManager$InvalidToken: Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1287) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:251) at org.apache.hadoop.hbase.client.HTable.(HTable.java:215) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:609) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Unknown master key for token (id=45 kinley/admin@KINLEY.COM) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:115) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.access$700(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:440) at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:511) at org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.java:69) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:900) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:164) at $Proxy12.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:215) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278) ... 21 more 2012-12-27 15:48:22,689 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2012-12-27 15:48:22,690 DEBUG org.apache.hadoop.ipc.Client: RPC Server's Kerberos principal name for protocol=org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB is hdfs/one.kinley.com@KINLEY.COM 2012-12-27 15:48:22,690 DEBUG org.apache.hadoop.ipc.Client: Use DIGEST authentication for protocol ClientNamenodeProtocolPB 2012-12-27 15:48:22,690 DEBUG org.apache.hadoop.ipc.Client: Connecting to one.kinley.com/172.16.223.3:8020 2012-12-27 15:48:22,690 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:585) 2012-12-27 15:48:22,690 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL DIGEST-MD5 client to authenticate to service at 172.16.223.3:8020 2012-12-27 15:48:22,691 DEBUG org.apache.hadoop.security.SaslRpcClient: Have sent token of size 0 from initSASLContext. 2012-12-27 15:48:22,691 DEBUG org.apache.hadoop.security.SaslRpcClient: Will read input token of size 108 for processing by initSASLContext 2012-12-27 15:48:22,691 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting username: ABdraW5sZXkvYWRtaW5AS0lOTEVZLkNPTQZtYXByZWQAigE73QtdfooBPAEX4X6OAUA8 2012-12-27 15:48:22,691 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting userPassword 2012-12-27 15:48:22,691 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting realm: default 2012-12-27 15:48:22,692 DEBUG org.apache.hadoop.security.SaslRpcClient: Will send token of size 310 from initSASLContext. 2012-12-27 15:48:22,692 DEBUG org.apache.hadoop.security.SaslRpcClient: Will read input token of size 40 for processing by initSASLContext 2012-12-27 15:48:22,693 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client context established. Negotiated QoP: auth 2012-12-27 15:48:22,693 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to one.kinley.com/172.16.223.3:8020 from kinley sending #0 2012-12-27 15:48:22,695 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to one.kinley.com/172.16.223.3:8020 from kinley: starting, having connections 1 2012-12-27 15:48:22,695 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to one.kinley.com/172.16.223.3:8020 from kinley got value #0 2012-12-27 15:48:22,695 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getBlockLocations took 9ms 2012-12-27 15:48:22,734 DEBUG org.apache.hadoop.hdfs.DFSClient: newInfo = LocatedBlocks{ fileLength=89 underConstruction=false blocks=[LocatedBlock{BP-1067390616-127.0.0.1-1343572746858:blk_8062484918615057032_84027; getBlockSize()=89; corrupt=false; offset=0; locs=[172.16.223.4:1004]}] lastLocatedBlock=LocatedBlock{BP-1067390616-127.0.0.1-1343572746858:blk_8062484918615057032_84027; getBlockSize()=89; corrupt=false; offset=0; locs=[172.16.223.4:1004]} isLastBlockComplete=true} 2012-12-27 15:48:22,735 DEBUG org.apache.hadoop.hdfs.DFSClient: Connecting to datanode 172.16.223.4:1004 2012-12-27 15:48:22,737 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to one.kinley.com/172.16.223.3:8020 from kinley sending #1 2012-12-27 15:48:22,738 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to one.kinley.com/172.16.223.3:8020 from kinley got value #1 2012-12-27 15:48:22,738 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getServerDefaults took 1ms 2012-12-27 15:48:22,791 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2012-12-27 15:48:22,793 DEBUG org.apache.hadoop.mapred.TaskLogsTruncater: Truncation is not needed for /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/stdout 2012-12-27 15:48:22,793 DEBUG org.apache.hadoop.mapred.TaskLogsTruncater: Truncation is not needed for /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/stderr 2012-12-27 15:48:22,793 DEBUG org.apache.hadoop.mapred.TaskLogsTruncater: Truncation is not needed for /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/syslog 2012-12-27 15:48:22,793 DEBUG org.apache.hadoop.mapred.TaskLogsTruncater: Cannot open /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/profile.out for reading. Continuing with other log files java.io.FileNotFoundException: /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/profile.out (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:120) at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:199) at org.apache.hadoop.mapred.Child$4.run(Child.java:273) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:22,793 DEBUG org.apache.hadoop.mapred.TaskLogsTruncater: Cannot open /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/debugout for reading. Continuing with other log files java.io.FileNotFoundException: /var/log/hadoop-0.20-mapreduce/userlogs/job_201212271546_0001/attempt_201212271546_0001_m_000000_0/debugout (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:120) at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:199) at org.apache.hadoop.mapred.Child$4.run(Child.java:273) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:22,793 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:kinley (auth:SIMPLE) cause:java.io.IOException: Cannot create a record reader because of a previous error. Please look at the previous logs lines from the task's full log for more details. 2012-12-27 15:48:22,793 WARN org.apache.hadoop.mapred.Child: Error running child java.io.IOException: Cannot create a record reader because of a previous error. Please look at the previous logs lines from the task's full log for more details. at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:98) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:617) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2012-12-27 15:48:22,794 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:kinley (auth:SIMPLE) from:org.apache.hadoop.mapred.Child.main(Child.java:297) 2012-12-27 15:48:22,794 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #27 2012-12-27 15:48:22,796 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #27 2012-12-27 15:48:22,796 DEBUG org.apache.hadoop.ipc.RPC: Call: statusUpdate 2 2012-12-27 15:48:22,796 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task 2012-12-27 15:48:22,796 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 sending #28 2012-12-27 15:48:22,797 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001 got value #28 2012-12-27 15:48:22,797 DEBUG org.apache.hadoop.ipc.RPC: Call: reportDiagnosticInfo 1 2012-12-27 15:48:22,797 DEBUG org.apache.hadoop.ipc.Client: Stopping client 2012-12-27 15:48:22,798 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001: closed 2012-12-27 15:48:22,798 DEBUG org.apache.hadoop.ipc.Client: IPC Client (918201446) connection to /127.0.0.1:59747 from job_201212271546_0001: stopped, remaining connections 0