Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.0.0-alpha-1
-
None
-
None
Description
This was caught by execution tests from our romil.choksi, when running hbase with WALs on hdfs, HFiles on S3. Upon initial analysis from elserj of below stack trace, we concluded this is triggered by enabling hbase.region.archive.recovered.edits property.
***** ABORTING region server my-rs-1.example.com,16020,1571335836583: Failed to open region hbase:meta,,1.1588230740 and can not recover ***** Cause: java.lang.IllegalArgumentException: Wrong FS: s3a://qe-s3-bucket-mst/cc-hbase-191017-113225-e6x-dw-4/hbase/archive/data/hbase/meta/1588230740/recovered.edits, expected: hdfs://my-hdfs-1.example.com:8020 at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:742) at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:234) at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1323) at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1320) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1337) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1312) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2292) at org.apache.hadoop.hbase.backup.HFileArchiver.archiveStoreFiles(HFileArchiver.java:321) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.removeStoreFiles(HRegionFileSystem.java:505) at org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:4626) at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:968) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:915) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7320)
This property is available mainly for debugging purposes, and it wraps recovered edits over store files for re-using the StoreFiles archiving logic.
Attachments
Attachments
Issue Links
- links to