[HBASE-12162] HBaseAdmin#getTableDescriptor() may fail in case master fails over

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 02/Oct/14 21:07
• Updated: 17/Jun/22 17:32
• Resolved: 01/Dec/14 17:49

Description

This was discovered by Chakradhar Medavarapu during HA testing.

Here is relevant exception:

2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException: Call to onprem-ha34/10.215.18.85:60000 failed on local exception: java.io.IOException: Call id=1, waitTime=8703
2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1541)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1723)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1776)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getTableDescriptors(MasterProtos.java:42525)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.getTableDescriptors(ConnectionManager.java:2121)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getHTableDescriptor(ConnectionManager.java:2600)
2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:410)

From stack trace, exception came out of connection.getHTableDescriptor().
This happened during master failover where MasterKeepAliveConnection to the failed master became unusable.

Issue Links

• is related to: HBASE-12072 Standardize retry handling for master operations