[HBASE-6416] hbck dies on NPE when a region folder exists but the table does not

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Later
• Fix Version/s: 2.0.0
• Assignee: Unassigned
• Reporter: Jean-Daniel Cryans
• Created: 17/Jul/12 23:18
• Updated: 13/Jun/22 17:04
• Resolved: 30/Jun/17 12:33

Description

This is what I'm getting for leftover data that has no .regioninfo

First:

12/07/17 23:13:37 WARN util.HBaseFsck: Failed to read .regioninfo file for region null
java.io.FileNotFoundException: File does not exist: /hbase/stumble_info_urlid_user/bd5f6cfed674389b4d7b8c1be227cb46/.regioninfo
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1822)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1813)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:544)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:456)
at org.apache.hadoop.hbase.util.HBaseFsck.loadHdfsRegioninfo(HBaseFsck.java:611)
at org.apache.hadoop.hbase.util.HBaseFsck.access$2200(HBaseFsck.java:140)
at org.apache.hadoop.hbase.util.HBaseFsck$WorkItemHdfsRegionInfo.call(HBaseFsck.java:2882)
at org.apache.hadoop.hbase.util.HBaseFsck$WorkItemHdfsRegionInfo.call(HBaseFsck.java:2866)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Then it hangs on:

12/07/17 23:13:39 INFO util.HBaseFsck: Attempting to handle orphan hdfs dir: hdfs://sfor3s24:10101/hbase/stumble_info_urlid_user/bd5f6cfed674389b4d7b8c1be227cb46
12/07/17 23:13:39 INFO util.HBaseFsck: checking orphan for table null
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hbase.util.HBaseFsck$TableInfo.access$100(HBaseFsck.java:1634)
at org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphan(HBaseFsck.java:435)
at org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphans(HBaseFsck.java:408)
at org.apache.hadoop.hbase.util.HBaseFsck.restoreHdfsIntegrity(HBaseFsck.java:529)
at org.apache.hadoop.hbase.util.HBaseFsck.offlineHdfsIntegrityRepair(HBaseFsck.java:313)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:386)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3227)

The NPE is sent by:

Preconditions.checkNotNull("Table " + tableName + "' not present!", tableInfo);

I wonder why the condition checking was added if we don't handle it... In any case hbck dies but it hangs because there are some non-daemon hanging around.