[HBASE-23553] Snapshot referenced data files are deleted in some case - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 3.0.0-alpha-1, 2.3.0, 2.2.3
Component/s: snapshots
Labels:
None

Description

We scan snapshot in our cluster and got following exception:

java.io.IOException: java.io.IOException: java.io.FileNotFoundException: Unable to open link: org.apache.hadoop.hbase.io.HFileLink locations=[hdfs://tjwqsrv-galaxy98/hbase/tjwqsrv-galaxy98/data/default/galaxy_online_fds_object_table/06dd90d8540b56343859b63a6134450c/A/4dddda6cf05f419a9f61059cb05a962f, hdfs://tjwqsrv-galaxy98/hbase/tjwqsrv-galaxy98/.tmp/data/default/galaxy_online_fds_object_table/06dd90d8540b56343859b63a6134450c/A/4dddda6cf05f419a9f61059cb05a962f, hdfs://tjwqsrv-galaxy98/hbase/tjwqsrv-galaxy98/archive/data/default/galaxy_online_fds_object_table/06dd90d8540b56343859b63a6134450c/A/4dddda6cf05f419a9f61059cb05a962f] 
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:867) 
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:778) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:749) 
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5306) 
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5271) 
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5243) 
at org.apache.hadoop.hbase.client.ClientSideRegionScanner.<init>(ClientSideRegionScanner.java:72) 
at org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatImpl$RecordReader.initialize(TableSnapshotInputFormatImpl.java:239) 
at org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat$TableSnapshotRegionRecordReader.initialize(TableSnapshotInputFormat.java:150) 
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:552) at

I checked to namenode logs and found that this file is deleted by hbase cleaner although a snapshot still referenced to this file.