Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18484

VerifyRep by snapshot does not work when Yarn / SourceHBase / PeerHBase located in different HDFS clusters

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      As HBASE-16466 commented.

      Seems like that when source hbase cluster / peer hbase cluster / yarn cluster locate in three different HDFS cluster , it has one problem.
      when restoring the snapshot into tmpdir , we need to create region by following code (HRegion#createHRegion)

      public static HRegion createHRegion(final HRegionInfo info, final Path rootDir,
              final Configuration conf, final TableDescriptor hTableDescriptor,
              final WAL wal, final boolean initialize)
        throws IOException {
          LOG.info("creating HRegion " + info.getTable().getNameAsString()
              + " HTD == " + hTableDescriptor + " RootDir = " + rootDir +
              " Table name == " + info.getTable().getNameAsString());
          FileSystem fs = FileSystem.get(conf);                                  <-------------------  Here our code use  fs.defaultFs configuration to create region.
          Path tableDir = FSUtils.getTableDir(rootDir, info.getTable());
          HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info);
          HRegion region = HRegion.newHRegion(tableDir, wal, fs, conf, info, hTableDescriptor, null);
          if (initialize) region.initialize(null);
          return region;
        }
      

      When source cluster & peer cluster locate in two difference file systems , then their fs.defaultFs should be difference, so at least one cluster will fail when restore snapshot into tmpdir . after I added the following fix, it works fine for me.

      -FileSystem fs = FileSystem.get(conf);  
      +FileSystem fs = rootDir.getFileSystem(conf);
      

      Attachments

        1. HBASE-18484.v3.patch
          13 kB
          Zheng Hu
        2. HBASE-18484.v2.patch
          13 kB
          Zheng Hu
        3. HBASE-18484.v1.patch
          1 kB
          Zheng Hu

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            openinx Zheng Hu
            openinx Zheng Hu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment