Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24471

The way we bootstrap meta table is confusing

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 2.4.0
    • Component/s: master, meta, proc-v2
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      Move all the meta initialization code in MasterFileSystem and HRegionServer to InitMetaProcedure. Add a new step for InitMetaProcedure called INIT_META_WRITE_FS_LAYOUT to place the moved code.

      This is an incompatible change, but should not have much impact. InitMetaProcedure will only be executed once when bootstraping a fresh new cluster, so typically this will not effect rolling upgrading. And even if you hit this problem, as long as InitMetaProcedure has not been finished, we can make sure that there is no user data in the cluster, you can just clean up the cluster and try again. There will be no data loss.
      Show
      Move all the meta initialization code in MasterFileSystem and HRegionServer to InitMetaProcedure. Add a new step for InitMetaProcedure called INIT_META_WRITE_FS_LAYOUT to place the moved code. This is an incompatible change, but should not have much impact. InitMetaProcedure will only be executed once when bootstraping a fresh new cluster, so typically this will not effect rolling upgrading. And even if you hit this problem, as long as InitMetaProcedure has not been finished, we can make sure that there is no user data in the cluster, you can just clean up the cluster and try again. There will be no data loss.

      Description

      I used to thought the work is only done in the InitMetaProcedure but in fact we have several pieces of code related to this which are placed in different classes.

      In HRegionServer.initializeFileSystem, we will test whether we are HMaster. If so, we will call FSTableDescriptors.tryUpdateMetaTableDescriptor. And in MasterFileSystem.checkRootDir, we will check if meta region is available. If not, we will bootstrap it, and it will also call FSTableDescriptors.tryUpdateMetaTableDescriptor.

      The problem here is that, HRegionServer.initializeFileSystem is called in the constructor of HRegionServer, which means it will be executed before MasterFileSystem.checkRootDir, as we will only create the MasterFileSystem for active master...

      Ideally I prefer we move all the bootstrap code into InitMetaProcedure, which will be much cleaner. If this can not be done, at least we could move the code into MasterFileSystem.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zhangduo Duo Zhang
                Reporter:
                zhangduo Duo Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: