From 0b7a375d4f5b76d7328499048ee8823f7b26a1de Mon Sep 17 00:00:00 2001 From: Sean Busbey Date: Thu, 22 Mar 2018 15:20:12 -0500 Subject: [PATCH] HBASE-19158 First pass at a 1.2 -> 2.0 upgrade section. --- src/main/asciidoc/_chapters/upgrading.adoc | 117 ++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 1 deletion(-) diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc index 0747ffbd71..8b8d0fd902 100644 --- a/src/main/asciidoc/_chapters/upgrading.adoc +++ b/src/main/asciidoc/_chapters/upgrading.adoc @@ -324,9 +324,124 @@ Quitting... == Upgrade Paths +[[upgrade2.0]] +=== Upgrading from 1.x to 2.x + +In this section we will first call out significant changes compared to the prior stable HBase release and then go over the upgrade process. Be sure to read the former with care so you avoid suprises. + +==== Changes of Note! + +First we'll cover deployment / operational changes that you might hit when upgrading to HBase 2.0+. After that we'll call out changes for downstream applications. Please note that Coprocessors are covered in the operational section. Also note that this section is not meant to convey information about new features that may be of interest to you. For a complete summary of changes, please see the CHANGES.txt file in the source release artifact for the version you are planning to upgrade to. + +[[upgrade2.0.basic.requirements]] +.Update to basic prerequisite minimums in HBase 2.0+ +As noted in the section [[basic.prerequisites]], HBase 2.0+ requires a minimum of Java 8 and Hadoop 2.6. The HBase community recommends using the latest stable release of Hadoop and upgrading prior to upgrading your HBase version. + +[[upgrade2.0.hbck]] +.HBCK must match HBase server version +You *must not* use an HBase 1.x version of HBCK against an HBase 2.0+ cluster. HBCK is strongly tied to the HBase server version. Using the HBCK tool from an earlier release against an HBase 2.0+ cluster will destructively alter said cluster in unrecoverable ways. + +As of HBase 2.0, HBCK is a read-only tool that can report the status of some non-public system internals. You should not rely on the format nor content of these internals to remain consistent across HBase releases. + +[[upgrade2.0.removed.configs]] + +The following configuration settings are no longer applicable or available. For details, please see the detailed release notes. + +* hbase.zookeeper.useMulti +* hbase.rpc.client.threads.max +* hbase.rpc.client.nativetransport +* hbase.fs.tmp.dir +* hbase.bucketcache.combinedcache.enabled +* hbase.bucketcache.ioengine no longer supports the 'heap' value. + +[[upgrade2.0.changed.defaults]] + +The following configuration settings changed their default value. Where applicable, the value to set to restore the behavior of HBase 1.2 is given. + +* hbase.security.authorization now defaults to false. set to true to restore same behavior as previous default. +* hbase.client.retries.number is now set to 10. Previously it was 35. Downstream users are advised to use client timeouts as described in section [[config_timeouts]] instead. +* hbase.client.serverside.retries.multiplier is now set to 3. Previously it was 10. Downstream users are advised to use client timesout as describe in section [[config_timeouts]] instead. +* hbase.master.fileSplitTimeout is now set to 10 minutes. Previously it was 30 seconds. +* hbase.regionserver.logroll.multiplier is now set to 0.5. Previously it was 0.95. +* hbase.regionserver.hlog.blocksize defaults to 2x the HDFS default block size for the WAL dir. Previously it was equal to the HDFS default block size for the WAL dir. +* hbase.client.start.log.errors.counter changed to 5. Previously it was 9. + +[[upgrade2.0.permissions]] + +The following permission related changes either altered semantics or defaults: + +* Permissions granted to a user now merge with existing permissions for that user, rather than over-writing them. (see [the release note on HBASE-17472](https://issues.apache.org/jira/browse/HBASE-17472)] for details) +* Region Server Group commands (added in 1.4.0) now require admin privileges. + +[[upgrade2.0.admin.commands]] +.Most Admin APIs don't work against an HBase 2.0+ cluster from pre-HBase 2.0 clients + +A number of admin commands are known to not work when used from a pre-HBase 2.0 client. This includes an HBase Shell that has the library jars from pre-HBase 2.0. You will need to plan for an outage of use of admin APIs and commands until you can also update to the needed client version. + +.Deprecated in 1.0 admin commands have been removed. + +The following commands that were deprecated in 1.0 have been removed. Where applicable the replacement command is listed. + +* The 'hlog' command has been removed. Downstream users should rely on the 'wal' command instead. + +[[upgrade2.0.replication]] +.Special upgrading for Replication users from pre-HBase 1.4 + +User running versions of HBase prior to the 1.4.0 release that make use of replication should be sure to read the instructions in the section [[upgrade1.4.replication]]. + +[[upgrade2.0.coprocessors]] +.Coprocessor APIs have changed in HBase 2.0+ + +All Coprocessor APIs have been refactored to improve supportability around binary API compatibility for future versions of HBase. If you or applications you rely on have custom HBase coprocessors, you should read [the release notes for HBASE-18169](https://issues.apache.org/jira/browse/HBASE-18169) for details of changes you will need to make prior to upgrading to HBase 2.0+. + +For example, if you had a BaseRegionObserver in HBase 1.2 then at a minimum you will need to update it to implement both RegionObserver and RegionCoprocessor and add the method + +[source,java] +---- +... + @Override + public Optional getRegionObserver() { + return Optional.of(this); + } +... +---- + +[[upgrade2.0.shaded.client.preferred]] +.Downstream HBase 2.0+ users should use the shaded client +Downstream users are strongly urged to rely on the Maven coordinates org.apache.hbase:hbase-shaded-client for their runtime use. This artifact contains all the needed implementation details for talking to an HBase cluster while minimizing the number of third party dependencies exposed. + +Note that this artifact exposes some classes in the org.apache.hadoop package space (e.g. o.a.h.configuration.Configuration) so that we can maintain source compatibility with our public API. Those classes are included so that they can be altered to use the same relocated third party dependencies as the rest of the HBase client code. In the event that you need to *also* use Hadoop in your code, you should ensure all Hadoop related jars precede the HBase client jar in your classpath. + +[[upgrade2.0.mapreduce.module]] +.Downstream HBase 2.0+ users of MapReduce must switch to new artifact +Downstream users of HBase's integration for Apache Hadoop MapReduce must switch to relying on the org.apache.hbase:hbase-shaded-mapreduce module for their runtime use. Historically, downstream users relied on either the org.apache.hbase:hbase-server or org.apache.hbase:hbase-shaded-server artifacts for these classes. Both uses are no longer supported and in the vast majority of cases will fail at runtime. + +Note that this artifact exposes some classes in the org.apache.hadoop package space (e.g. o.a.h.configuration.Configuration) so that we can maintain source compatibility with our public API. Those classes are included so that they can be altered to use the same relocated third party dependencies as the rest of the HBase client code. In the event that you need to *also* use Hadoop in your code, you should ensure all Hadoop related jars precede the HBase client jar in your classpath. + +[[upgrade2.0.dependencies]] +.Significant changes to runtime classpath +A number of internal dependencies for HBase were updated or removed from the runtime classpath. Downstream client users who do not follow the guidance in [[2.0.shaded.client.preferred]] will have to examine the set of dependencies Maven pulls in for impact. Downstream users of LimitedPrivate Coprocessor APIs will need to examine the runtime environment for impact. For details on our new handling of third party libraries that have historically been a problem with respect to harmonizing compatible runtime versions, see the reference guide section [[thirdparty]]. + +[[upgrade2.0.public.api]] +.Multiple breaking changes to source and binary compatibility for client API +The Java client API for HBase has a number of changes that break both source and binary compatibility for details see the Compatibility Check Report for the release you'll be upgrading to. + +[[upgrade2.0.rolling.upgrades]] +==== Rolling Upgrade from 1.x to 2.x +There is no rolling upgrade from HBase 1.x+ to HBase 2.x+. In order to perform a zero downtime upgrade, you will need to run an additional cluster in parallel and handle failover in application logic. + +[[upgrade2.0.process]] +==== Upgrade process from 1.2 to 2.x + +* Clean shutdown +* Upgrade Master roles first +* Upgrade RegionServers +* (Eventually) upgrade Clients + [[upgrade1.4]] -=== Upgrading to 1.4+ +=== Upgrading from pre-1.4 to 1.4+ +[[upgrade1.4.replication]] ==== Replication peer's TableCFs config Before 1.4, the table name can't include namespace for replication peer's TableCFs config. It was fixed by add TableCFs to ReplicationPeerConfig which was stored on Zookeeper. So when upgrade to 1.4, you have to update the original ReplicationPeerConfig data on Zookeeper firstly. There are four steps to upgrade when your cluster have a replication peer with TableCFs config. -- 2.16.1