Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15331

HBase Backup/Restore Phase 2: Optimized Restore operation

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      The current implementation for restore uses WALReplay M/R job. This has performance and stability problems, since it uses HBase client API to insert data. We have to migrate to bulk load approach: generate hfiles directly from snapshot and incremental images. We run separate M/R job for every backup image between last FULL backup and current incremental backup we restore to and for every table in the list (image). If we have 10 tables and 30 days of incremental backup images - this results in 30x10 = 300 M/R jobs. MUST be optimized.

      Attachments

        Issue Links

          is part of

          Umbrella - An overarching type made of sub-tasks HBASE-14123 HBase Backup/Restore Phase 2

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Task - A task that needs to be done. HBASE-15448 HBase Backup Phase 3: Restore optimization 2

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          requires

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-15372 HBase Backup/Restore Phase 2: Keep WAL files in a centralized place to avoid duplication

          • Major - Major loss of function.
          • Closed

          Activity

            People

              vrodionov Vladimir Rodionov
              vrodionov Vladimir Rodionov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: