Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22514

Reimplement rsgroup feature and move it into core of HBase

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: Admin, Client, rsgroup
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      Moved rs group feature into core. Use this flag 'hbase.balancer.rsgroup.enabled' to enable or disable it.

      The coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint is deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell to communicate with the new hbase cluster, you still need to add this coprocessor to master. And if this coprocessor is specified, the above flag will be set to true automatically to enable rs group feature.

      These methods are added to the Admin/AsyncAdmin interface for managing rs groups. See the javadoc of these methods for more details.

        void addRSGroup(String groupName) throws IOException;
        RSGroupInfo getRSGroup(String groupName) throws IOException;
        RSGroupInfo getRSGroup(Address hostPort) throws IOException;
        RSGroupInfo getRSGroup(TableName tableName) throws IOException;
        List<RSGroupInfo> listRSGroups() throws IOException;
        List<TableName> listTablesInRSGroup(String groupName) throws IOException;
        Pair<List<String>, List<TableName>> getConfiguredNamespacesAndTablesInRSGroup(String groupName) throws IOException;
        void removeRSGroup(String groupName) throws IOException;
        void removeServersFromRSGroup(Set<Address> servers) throws IOException;
        void moveServersToRSGroup(Set<Address> servers, String targetGroup) throws IOException;
        void setRSGroup(Set<TableName> tables, String groupName) throws IOException;
        boolean balanceRSGroup(String groupName) throws IOException;

      The shell commands for rs group are not changed.

      The main difference on the implementation is that, now the rs group for a table is stored in TableDescriptor, instead of in RSGroupInfo, so the getTables method of RSGroupInfo has been deprecated. And if you use the above Admin methods to get the RSGroupInfo, its getTables method will always return empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, we will fill the tables field of the RSGroupInfo before returning, to make it compatible with old hbase client/shell.

      When upgrading, the migration between the RSGroupInfo and TableDescriptor will be done automatically. It will take sometime, but it is fine to restart master in the middle, the migration will continue after restart.

      Show
      Moved rs group feature into core. Use this flag 'hbase.balancer.rsgroup.enabled' to enable or disable it. The coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint is deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell to communicate with the new hbase cluster, you still need to add this coprocessor to master. And if this coprocessor is specified, the above flag will be set to true automatically to enable rs group feature. These methods are added to the Admin/AsyncAdmin interface for managing rs groups. See the javadoc of these methods for more details.   void addRSGroup(String groupName) throws IOException;   RSGroupInfo getRSGroup(String groupName) throws IOException;   RSGroupInfo getRSGroup(Address hostPort) throws IOException;   RSGroupInfo getRSGroup(TableName tableName) throws IOException;   List<RSGroupInfo> listRSGroups() throws IOException;   List<TableName> listTablesInRSGroup(String groupName) throws IOException;   Pair<List<String>, List<TableName>> getConfiguredNamespacesAndTablesInRSGroup(String groupName) throws IOException;   void removeRSGroup(String groupName) throws IOException;   void removeServersFromRSGroup(Set<Address> servers) throws IOException;   void moveServersToRSGroup(Set<Address> servers, String targetGroup) throws IOException;   void setRSGroup(Set<TableName> tables, String groupName) throws IOException;   boolean balanceRSGroup(String groupName) throws IOException; The shell commands for rs group are not changed. The main difference on the implementation is that, now the rs group for a table is stored in TableDescriptor, instead of in RSGroupInfo, so the getTables method of RSGroupInfo has been deprecated. And if you use the above Admin methods to get the RSGroupInfo, its getTables method will always return empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, we will fill the tables field of the RSGroupInfo before returning, to make it compatible with old hbase client/shell. When upgrading, the migration between the RSGroupInfo and TableDescriptor will be done automatically. It will take sometime, but it is fine to restart master in the middle, the migration will continue after restart.

      Description

      The class RSGroupAdminClient is not public 

      we need to use java api  RSGroupAdminClient  to manager RSG 

      so  RSGroupAdminClient should be public

       

        Attachments

        1. HBASE-22514.master.001.patch
          1 kB
          Yechao Chen
        2. image-2019-05-31-18-25-38-217.png
          12 kB
          Yechao Chen
        1.
        Move RSGroupInfoManager to hbase-server Sub-task Resolved Duo Zhang
        2.
        Move protobuf stuff in hbase-rsgroup to hbase-protocol-shaded Sub-task Resolved Duo Zhang
        3.
        Move all the code in hbase-rsgroup to hbase-server and remove hbase-rsgroup module Sub-task Resolved Duo Zhang
        4.
        Store the rsgroup of a table in table configuration Sub-task Resolved Duo Zhang
        5.
        Start RSGroupInfoManager as default Sub-task Resolved Duo Zhang
        6.
        Allow creating table in group when rs group contains no live servers Sub-task Resolved Duo Zhang
        7.
        Revisit the logic of when we can remove the last server from a rsgroup and when can we remove a rsgroup Sub-task Resolved Unassigned
        8.
        Automatically migrate the rs group config for table after HBASE-22695 Sub-task Resolved Duo Zhang
        9.
        Do not need to persist default rs group now Sub-task Resolved Xiaolin Ha
        10.
        Add rs group management methods in Admin and AsyncAdmin Sub-task Resolved Xiaolin Ha
        11.
        Deprecated RSGroupAdminEndpoint and make RSGroup feature always enabled Sub-task Resolved Duo Zhang
        12.
        Calculate the region servers in default group in foreground Sub-task Resolved Duo Zhang
        13.
        Use RSGroupInfoManager to get rsgroups in master UI's rsgroup part Sub-task Resolved Guangxu Cheng
        14.
        Add an option to enable/disable rs group feature Sub-task Resolved Duo Zhang
        15.
        Remove rsgroup profile from pom.xml of hbase-assembly Sub-task Resolved Guangxu Cheng
        16.
        Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion Sub-task Resolved Duo Zhang
        17.
        Rewrite rsgroup related UTs with the new methods introduced in HBASE-22932 Sub-task Resolved Duo Zhang
        18.
        Add admin methods to get tables within a group Sub-task Resolved Duo Zhang
        19.
        Make rsgroup related shell command to use the new admin methods Sub-task Resolved Duo Zhang
        20.
        Cleanup the remaining RSGroupInfo.getTables call in the code base Sub-task Resolved Duo Zhang
        21.
        Update the rsgroup section in our ref guide Sub-task Resolved Duo Zhang
        22.
        Attach the new rsgroup implementation design doc to our code base Sub-task Resolved Duo Zhang

          Activity

            People

            • Assignee:
              zhangduo Duo Zhang
              Reporter:
              chenyechao Yechao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: