[HBASE-26898] Cannot rebuild a cluster from an existing root directory

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-2
• Fix Version/s: 3.0.0-alpha-2
• Component/s: master
• Assignee: Liangjun He
• Reporter: Liangjun He
• Created: 28/Mar/22 16:16
• Updated: 25/Oct/22 10:04
• Resolved: 25/Oct/22 10:04

Description

When I tested to rebuild an HBase cluster, and the rootdir was configured as a existed directory (the directory was generated by another HBase cluster of the same version), I found the following error message：

java.net.UnknownHostException: Call to address=worker-1.cluster-xxx:16020 failed on local exception: java.net.UnknownHostException: worker-1.cluster-xxx:16020 could not be resolved
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:234)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:387)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
    at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:119)
    at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:134)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection.lambda$sendRequest$4(NettyRpcConnection.java:351)
    at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
    at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
    at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: worker-1.cluster-xxx:16020 could not be resolved
    at org.apache.hadoop.hbase.ipc.RpcConnection.getRemoteInetAddress(RpcConnection.java:192)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection.connect(NettyRpcConnection.java:275)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$800(NettyRpcConnection.java:78)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection$4.run(NettyRpcConnection.java:325)
    at org.apache.hadoop.hbase.ipc.HBaseRpcControllerImpl.notifyOnCancel(HBaseRpcControllerImpl.java:262)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection.sendRequest0(NettyRpcConnection.java:308)
    at org.apache.hadoop.hbase.ipc.NettyRpcConnection.lambda$sendRequest$4(NettyRpcConnection.java:349) 

Eventually, I fail to create the cluster.

But for cloud environments, this operation is a common scenario（Rebuild a cluster from an existing rootdir directory）

 

 

Issue Links

• Blocked: HBASE-26914 Rebuild a cluster from an existing root directory may hang
• is related to: HBASE-26245 Store region server list in master local region