[HBASE-18484] VerifyRep by snapshot does not work when Yarn / SourceHBase / PeerHBase located in different HDFS clusters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha-1
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.0.5, 2.3.0, 2.1.4
• Component/s: Replication
• Hadoop Flags: Reviewed
• Assignee: Zheng Hu
• Reporter: Zheng Hu
• Created: 31/Jul/17 07:36
• Updated: 12/Feb/19 16:22
• Resolved: 12/Feb/19 03:21

Description

As HBASE-16466 commented.

Seems like that when source hbase cluster / peer hbase cluster / yarn cluster locate in three different HDFS cluster , it has one problem.
when restoring the snapshot into tmpdir , we need to create region by following code (HRegion#createHRegion)

public static HRegion createHRegion(final HRegionInfo info, final Path rootDir,
        final Configuration conf, final TableDescriptor hTableDescriptor,
        final WAL wal, final boolean initialize)
  throws IOException {
    LOG.info("creating HRegion " + info.getTable().getNameAsString()
        + " HTD == " + hTableDescriptor + " RootDir = " + rootDir +
        " Table name == " + info.getTable().getNameAsString());
    FileSystem fs = FileSystem.get(conf);                                  <-------------------  Here our code use  fs.defaultFs configuration to create region.
    Path tableDir = FSUtils.getTableDir(rootDir, info.getTable());
    HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info);
    HRegion region = HRegion.newHRegion(tableDir, wal, fs, conf, info, hTableDescriptor, null);
    if (initialize) region.initialize(null);
    return region;
  }

When source cluster & peer cluster locate in two difference file systems , then their fs.defaultFs should be difference, so at least one cluster will fail when restore snapshot into tmpdir . after I added the following fix, it works fine for me.

-FileSystem fs = FileSystem.get(conf);  
+FileSystem fs = rootDir.getFileSystem(conf);

Issue Links

• is related to: HBASE-18452 VerifyReplication by Snapshot should cache HDFS token before submit job for kerberos env.
• is related to: HBASE-16466 HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
• links to: Review board