[HBASE-16466] HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.98.21
• Fix Version/s: 2.0.0

Release Note

Support for snapshots in VerifyReplication tool i.e. verifyrep can compare source table snapshot against peer table snapshot which reduces load on RS by reading data from HDFS directly using Snapshot scanners.
Instead of comparing against live tables whose state changes due to writes and compactions its better to compare HBase snapshots which are immutable in nature.

• Assignee: Maddineni Sukumar
• Reporter: Sukumar Maddineni
• Created: 22/Aug/16 18:20
• Updated: 18/Jun/18 15:08
• Resolved: 03/May/17 23:03

Description

As of now VerifyReplicatin tool is running using normal HBase scanners. If you want to run VerifyReplication multiple times on a production live cluster with large tables then it creates extra load on HBase layer. So if we implement snapshot based support then both in source and target we can read data from snapshots which reduces load on HBase

Issue Links

• incorporates: HBASE-17899 VerifyReplication not exiting for invalid arguments even though we have check for it
• incorporates: HBASE-17900 VerifyReplication - input param variables declared as static causing issues to run VerifyReplication multiple times in single JVM(mainly for unit tests)
• relates to: HBASE-18452 VerifyReplication by Snapshot should cache HDFS token before submit job for kerberos env.
• relates to: HBASE-18484 VerifyRep by snapshot does not work when Yarn / SourceHBase / PeerHBase located in different HDFS clusters