From 1b95bf5dfd0b12468763e1d6d4af3cb7f06cc415 Mon Sep 17 00:00:00 2001 From: Josh Elser Date: Mon, 27 Mar 2017 16:55:31 -0400 Subject: [PATCH] HBASE-17840 Update hbase book to space quotas on snapshots --- src/main/asciidoc/_chapters/ops_mgt.adoc | 77 ++++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 19 deletions(-) diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc b/src/main/asciidoc/_chapters/ops_mgt.adoc index f60395bbc5..a4527d8dda 100644 --- a/src/main/asciidoc/_chapters/ops_mgt.adoc +++ b/src/main/asciidoc/_chapters/ops_mgt.adoc @@ -1965,6 +1965,48 @@ automatically in hbase-site.xml. ---- +=== HBase Snapshots with Space Quotas + +One common area of unintended-filesystem-use with HBase is via HBase snapshots. Because snapshots +exist outside of the management of HBase tables, it is not uncommon for administrators to suddenly +realize that hundreds of gigabytes or terabytes of space is being used by HBase snapshots which were +forgotten and never removed. + +link:https://issues.apache.org/jira/browse/HBASE-17748[HBASE-17748] is the umbrella JIRA issue which +expands on the original space quota functionality to also include HBase snapshots. While this is a confusing +subject, the implementation attempts to present this support in as reasonable/simple of a manner as +possible for administrators. This feature does not make any changes to administrator interaction with +space quotas, only in the internal computation of table/namespace usage. Table and namespace usage will +automatically incorporate the size taken by a snapshot per the rules defined below. + +As a review, let's cover a snapshot's lifecycle. A snapshot is metadata which points to +a list of HFiles on the filesystem. This is why creating a snapshot is a very cheap operation; no HBase +table data is actually copied to perform a snapshot. Cloning a snapshot into a new table or restoring +a table is a cheap operation for the same reason; the new table references the files which already exist +on the filesystem without a copy. To include snapshots in space quotas, we need to define what table +"owns" a file when a snapshot references the file ("owns" refers to encompassing the filesystem usage +of that file). + +When a snapshot refers to a file and no table refers to that file, the table from which that snapshot was +created "owns" that file. When multiple snapshots refer to the same file and no table refers to that file, +the snapshot with the lowest-sorting name (lexicographically) is chosen and the table which that snapshot +was created from "owns" that file. HFiles are not "double-counted" when a table and one or more snapshots +refer to that HFile. + +When a table is "rematerialized" (via `clone_snapshot` or `restore_snapshot`), a similar problem of file +"ownership" arises. In this case, while the rematerialized table references a file which a snapshot also +references, the table does not "own" the file. The table from which the snapshot was created still "owns" +that file. When the rematerialized table is compacted or the snapshot is deleted, the rematerialized table +will uniquely refer to a new file and "own" the usage of that file. + +One new HBase shell command was added to inspect the computed sizes of each snapshot in an HBase instance. + +--- +hbase> list_snapshot_sizes +SNAPSHOT SIZE + t1.s1 1159108 +--- + [[ops.backup]] == HBase Backup @@ -2434,7 +2476,7 @@ void rename(Admin admin, String oldTableName, TableName newTableName) { RegionServer Grouping (A.K.A `rsgroup`) is an advanced feature for partitioning regionservers into distinctive groups for strict isolation. It should only be used by users who are sophisticated enough to understand the -full implications and have a sufficient background in managing HBase clusters. +full implications and have a sufficient background in managing HBase clusters. It was developed by Yahoo! and they run it at scale on their large grid cluster. See link:http://www.slideshare.net/HBaseCon/keynote-apache-hbase-at-yahoo-scale[HBase at Yahoo! Scale]. @@ -2447,20 +2489,20 @@ rsgroup at a time. By default, all tables and regionservers belong to the APIs. A custom balancer implementation tracks assignments per rsgroup and makes sure to move regions to the relevant regionservers in that rsgroup. The rsgroup information is stored in a regular HBase table, and a zookeeper-based read-only -cache is used at cluster bootstrap time. +cache is used at cluster bootstrap time. -To enable, add the following to your hbase-site.xml and restart your Master: +To enable, add the following to your hbase-site.xml and restart your Master: [source,xml] ---- - - hbase.coprocessor.master.classes - org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint - - - hbase.master.loadbalancer.class - org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer - + + hbase.coprocessor.master.classes + org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint + + + hbase.master.loadbalancer.class + org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer + ---- Then use the shell _rsgroup_ commands to create and manipulate RegionServer @@ -2470,7 +2512,7 @@ rsgroup commands available in the hbase shell type: [source, bash] ---- hbase(main):008:0> help ‘rsgroup’ - Took 0.5610 seconds + Took 0.5610 seconds ---- High level, you create a rsgroup that is other than the `default` group using @@ -2487,8 +2529,8 @@ Here is example using a few of the rsgroup commands. To add a group, do as foll [source, bash] ---- - hbase(main):008:0> add_rsgroup 'my_group' - Took 0.5610 seconds + hbase(main):008:0> add_rsgroup 'my_group' + Took 0.5610 seconds ---- @@ -2512,11 +2554,11 @@ ERROR: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registere ==== Add a server (specified by hostname + port) to the just-made group using the -_move_servers_rsgroup_ command as follows: +_move_servers_rsgroup_ command as follows: [source, bash] ---- - hbase(main):010:0> move_servers_rsgroup 'my_group',['k.att.net:51129'] + hbase(main):010:0> move_servers_rsgroup 'my_group',['k.att.net:51129'] ---- .Hostname and Port vs ServerName @@ -2573,6 +2615,3 @@ before you start moving out the dead. Move in good live nodes first if you have Viewing the Master log will give you insight on rsgroup operation. If it appears stuck, restart the Master process. - - - -- 2.12.2