[HBASE-9313] NamespaceJanitor is spammy when the namespace table moves

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.95.2
• Fix Version/s: 0.98.0, 0.96.0
• Hadoop Flags: Reviewed
• Assignee: Jean-Daniel Cryans
• Reporter: Jean-Daniel Cryans
• Created: 22/Aug/13 22:56
• Updated: 20/Nov/15 11:52
• Resolved: 28/Aug/13 19:17

Description

Although region movements are part of a healthy HBase lifestyle, the NamespaceJanitor WARNs about it:

2013-08-22 22:35:48,872 WARN  [NamespaceJanitor-jdec2hbase0403-1:60000] org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, tries=0, retries=350, retryTime=-4ms
org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 640d4b4d9432f23f1638700217d34764
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:235)
	at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:300)
	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:148)
	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57)
	at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
	at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:98)
	at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:239)
	at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:153)
	at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:100)
	at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:696)
	at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:707)
	at org.apache.hadoop.hbase.master.TableNamespaceManager.list(TableNamespaceManager.java:185)
	at org.apache.hadoop.hbase.master.HMaster.listNamespaceDescriptors(HMaster.java:3149)
	at org.apache.hadoop.hbase.master.NamespaceJanitor.removeOrphans(NamespaceJanitor.java:102)
	at org.apache.hadoop.hbase.master.NamespaceJanitor.chore(NamespaceJanitor.java:86)
	at org.apache.hadoop.hbase.Chore.run(Chore.java:80)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 640d4b4d9432f23f1638700217d34764
	at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2565)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3927)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3004)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26847)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2156)
	at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)

	at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1426)
	at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1630)
	at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1687)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:27303)
	at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:291)
	... 15 more

This should not be printed.