[HBASE-13280] TestSecureRPC failed

• Type: Test
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.1.0, 2.0.0
• Fix Version/s: 1.1.0, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Shaohui Liu
• Created: 19/Mar/15 03:35
• Updated: 03/Jun/15 15:29
• Resolved: 19/Mar/15 13:50

Description

Running org.apache.hadoop.hbase.security.TestSecureRPC
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 33.795 sec <<< FAILURE! - in org.apache.hadoop.hbase.security.TestSecureRPC
testRpc(org.apache.hadoop.hbase.security.TestSecureRPC)  Time elapsed: 14.963 sec  <<< ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:635)
	at java.util.ArrayList.get(ArrayList.java:411)
	at org.apache.hadoop.hbase.security.TestSecureRPC.testRpcCallWithEnabledKerberosSaslAuth(TestSecureRPC.java:160)
	at org.apache.hadoop.hbase.security.TestSecureRPC.testRpc(TestSecureRPC.java:102)

testAsyncRpc(org.apache.hadoop.hbase.security.TestSecureRPC)  Time elapsed: 5.15 sec  <<< ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:635)
	at java.util.ArrayList.get(ArrayList.java:411)
	at org.apache.hadoop.hbase.security.TestSecureRPC.testRpcCallWithEnabledKerberosSaslAuth(TestSecureRPC.java:160)
	at org.apache.hadoop.hbase.security.TestSecureRPC.testAsyncRpc(TestSecureRPC.java:107)

From log we saw that:

2015-03-19 11:27:02,271 WARN  [Thread-5] ipc.RpcClientImpl$Connection$1(662): Couldn't setup connection for hbase/[e-mail] to hbase/[e-mail]
Exception in thread "Thread-5" java.lang.RuntimeException: com.google.protobuf.ServiceException: java.io.IOException: Couldn't setup connection for hbase/[e-mail] to hbase/[e-mail]
	at org.apache.hadoop.hbase.ipc.TestDelayedRpc$TestThread.run(TestDelayedRpc.java:275)
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Couldn't setup connection for hbase/[e-mail] to hbase/[e-mail]
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
	at org.apache.hadoop.hbase.ipc.protobuf.generated.TestDelayedRpcProtos$TestDelayedService$BlockingStub.test(TestDelayedRpcProtos.java:1115)
	at org.apache.hadoop.hbase.ipc.TestDelayedRpc$TestThread.run(TestDelayedRpc.java:272)
Caused by: java.io.IOException: Couldn't setup connection for hbase/[e-mail] to hbase/[e-mail]
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:663)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1170)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
	... 3 more
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]
	at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
	at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
	... 7 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)
	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
	at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
	... 16 more
Caused by: KrbException: Server not found in Kerberos database (7) - Server not found in Kerberos database
	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
	at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
	at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
	at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:311)
	at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
	at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:449)
	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
	... 19 more
Caused by: KrbException: Identifier doesn't match expected value (906)
	at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143)
	at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66)
	at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61)
	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
	... 25 more

Any idea about Apache9? Thanks