From a1c5d22cee1be308df2535c2211b4412355121ac Mon Sep 17 00:00:00 2001 From: Wellington Chevreuil Date: Mon, 12 Aug 2019 19:43:53 +0100 Subject: [PATCH] HBASE-22837 Move Custom WAL Directory section from Bulk Loading to Write Ahead Log chapter --- src/main/asciidoc/_chapters/architecture.adoc | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc index 886ac08b92..a2fe2f1e90 100644 --- a/src/main/asciidoc/_chapters/architecture.adoc +++ b/src/main/asciidoc/_chapters/architecture.adoc @@ -1266,6 +1266,26 @@ It is possible to set _durability_ on each Mutation or on a Table basis. Options Do not confuse the _ASYNC_WAL_ option on a Mutation or Table with the _AsyncFSWAL_ writer; they are distinct options unfortunately closely named +[[arch.custom.wal.dir]] +==== Custom WAL Directory +HBASE-17437 added support for specifying a WAL directory outside the HBase root directory or even in a different FileSystem since 1.3.3/2.0+. Some FileSystems (such as Amazon S3) don’t support append or consistent writes, in such scenario WAL directory needs to be configured in a different FileSystem to avoid loss of writes. + +Following configurations are added to accomplish this: + +. `hbase.wal.dir` ++ +This defines where the root WAL directory is located, could be on a different FileSystem than the root directory. WAL directory can not be set to a subdirectory of the root directory. The default value of this is the root directory if unset. + +. `hbase.rootdir.perms` ++ +Configures FileSystem permissions to set on the root directory. This is '700' by default. + +. `hbase.wal.dir.perms` ++ +Configures FileSystem permissions to set on the WAL directory FileSystem. This is '700' by default. + +NOTE: While migrating to custom WAL dir (outside the HBase root directory or a different FileSystem) existing WAL files must be copied manually to new WAL dir, otherwise it may lead to data loss/inconsistency as HMaster has no information about previous WAL directory. + [[wal.disable]] ==== Disabling the WAL @@ -2491,26 +2511,6 @@ For example: If source cluster FS client configurations are copied to the destin NOTE: `DefaultSourceFSConfigurationProvider` supports only `xml` type files. It loads source cluster FS client configuration only once, so if source cluster FS client configuration files are updated, every peer(s) cluster RS must be restarted to reload the configuration. -[[arch.custom.wal.dir]] -=== Custom WAL Directory -HBASE-17437 added support for specifying a WAL directory outside the HBase root directory or even in a different FileSystem since 1.3.3/2.0+. Some FileSystems (such as Amazon S3) don’t support append or consistent writes, in such scenario WAL directory needs to be configured in a different FileSystem to avoid loss of writes. - -Following configurations are added to accomplish this: - -. `hbase.wal.dir` -+ -This defines where the root WAL directory is located, could be on a different FileSystem than the root directory. WAL directory can not be set to a subdirectory of the root directory. The default value of this is the root directory if unset. - -. `hbase.rootdir.perms` -+ -Configures FileSystem permissions to set on the root directory. This is '700' by default. - -. `hbase.wal.dir.perms` -+ -Configures FileSystem permissions to set on the WAL directory FileSystem. This is '700' by default. - -NOTE: While migrating to custom WAL dir (outside the HBase root directory or a different FileSystem) existing WAL files must be copied manually to new WAL dir, otherwise it may lead to data loss/inconsistency as HMaster has no information about previous WAL directory. - [[arch.hdfs]] == HDFS -- 2.17.2 (Apple Git-113)