XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha-1, 2.2.0
    • 3.0.0-alpha-1, 2.2.0
    • None
    • None
    • Hide
      After HBASE-21588, we introduce a new way to do WAL splitting coordination by procedure framework. This can simplify the process of WAL splitting and no need to connect zookeeper any more.
      During ServerCrashProcedure, it will create a SplitWALProcedure for each WAL that needs to split. Then each SplitWALProcedure will spawn a SplitWALRemoteProcedure to send the request to regionserver.
      At the RegionServer side, whole split process is handled by a SplitWALCallable. It splits the WAL and returns the result to the master.
      According to my test, this patch has a better performance as the number of WALs that need to split increase. And it can relieve the pressure on zookeeper.

      Set 'hbase.split.wal.zk.coordinated' to false to 'enable'. 'hbase.regionserver.wal.max.splitters' is reused to set how many 'splitters' to run on the RegionServer side.
      Show
      After HBASE-21588 , we introduce a new way to do WAL splitting coordination by procedure framework. This can simplify the process of WAL splitting and no need to connect zookeeper any more. During ServerCrashProcedure, it will create a SplitWALProcedure for each WAL that needs to split. Then each SplitWALProcedure will spawn a SplitWALRemoteProcedure to send the request to regionserver. At the RegionServer side, whole split process is handled by a SplitWALCallable. It splits the WAL and returns the result to the master. According to my test, this patch has a better performance as the number of WALs that need to split increase. And it can relieve the pressure on zookeeper. Set 'hbase.split.wal.zk.coordinated' to false to 'enable'. 'hbase.regionserver.wal.max.splitters' is reused to set how many 'splitters' to run on the RegionServer side.

    Description

      create a sub task to submit the implementation of procedure v2 wal splitting

      Attachments

        1. HBASE-21588.branch-2.002.patch
          93 kB
          Jingyun Tian
        2. HBASE-21588.branch-2.001.patch
          93 kB
          Jingyun Tian
        3. HBASE-21588.master.020.patch
          89 kB
          Jingyun Tian
        4. HBASE-21588.master.019.patch
          89 kB
          Michael Stack
        5. HBASE-21588.master.019.patch
          89 kB
          Jingyun Tian
        6. HBASE-21588.master.018.patch
          88 kB
          Michael Stack
        7. HBASE-21588.master.018.patch
          88 kB
          Jingyun Tian
        8. HBASE-21588.master.017.patch
          87 kB
          Jingyun Tian
        9. HBASE-21588.master.016.patch
          87 kB
          Jingyun Tian
        10. HBASE-21588.master.015.patch
          87 kB
          Jingyun Tian
        11. HBASE-21588.master.014.patch
          85 kB
          Jingyun Tian
        12. HBASE-21588.master.013.patch
          87 kB
          Jingyun Tian
        13. HBASE-21588.master.012.patch
          86 kB
          Jingyun Tian
        14. HBASE-21588.master.011.patch
          87 kB
          Jingyun Tian
        15. HBASE-21588.master.010.patch
          87 kB
          Jingyun Tian
        16. HBASE-21588.master.009.patch
          95 kB
          Jingyun Tian
        17. HBASE-21588.master.008.patch
          82 kB
          Jingyun Tian
        18. HBASE-21588.master.007.patch
          84 kB
          Jingyun Tian
        19. HBASE-21588.master.006.patch
          84 kB
          Jingyun Tian
        20. HBASE-21588.master.005.patch
          93 kB
          Jingyun Tian
        21. HBASE-21588.master.004.patch
          93 kB
          Jingyun Tian
        22. HBASE-21588.master.003.patch
          93 kB
          Jingyun Tian

        Issue Links

          Activity

            People

              tianjingyun Jingyun Tian
              tianjingyun Jingyun Tian
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: