Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16466

HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.98.21
    • 2.0.0
    • None
    • None
    • Hide
      Support for snapshots in VerifyReplication tool i.e. verifyrep can compare source table snapshot against peer table snapshot which reduces load on RS by reading data from HDFS directly using Snapshot scanners.
      Instead of comparing against live tables whose state changes due to writes and compactions its better to compare HBase snapshots which are immutable in nature.
      Show
      Support for snapshots in VerifyReplication tool i.e. verifyrep can compare source table snapshot against peer table snapshot which reduces load on RS by reading data from HDFS directly using Snapshot scanners. Instead of comparing against live tables whose state changes due to writes and compactions its better to compare HBase snapshots which are immutable in nature.

    Description

      As of now VerifyReplicatin tool is running using normal HBase scanners. If you want to run VerifyReplication multiple times on a production live cluster with large tables then it creates extra load on HBase layer. So if we implement snapshot based support then both in source and target we can read data from snapshots which reduces load on HBase

      Attachments

        1. HBASE-16466.v5.patch
          25 kB
          Maddineni Sukumar
        2. HBASE-16466.v4.patch
          28 kB
          Maddineni Sukumar
        3. HBASE-16466.v3.patch
          28 kB
          Maddineni Sukumar
        4. HBASE-16466.v2.patch
          46 kB
          Maddineni Sukumar
        5. HBASE-16466.v1.patch
          28 kB
          Maddineni Sukumar
        6. HBASE-16466.branch-1.3.001.patch
          26 kB
          Maddineni Sukumar

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sukunaidu@gmail.com Maddineni Sukumar
            sukumaddineni Sukumar Maddineni
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment