Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3285

Hlog recovery takes too much time

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.90.2
    • None
    • None
    • Adds hbase exploitation of new lease recovery added by hdfs-1520. New API is available at tip of branch-0.20-append, in advance of the version of hadoop that ships with 0.90.1 hbase (r1057313). Must patch CDH to add the API.

    Description

      Currently HBase uses append to trigger the close of HLog during Hlog split. Append is a very expensive operation, which involves not only NameNode operations but creating a writing pipeline. If one of datanodes on the pipeline has a problem, this recovery may takes minutes. I'd like implement a lightweight NameNode operation to trigger lease recovery and make HBase to use this instead.

      Attachments

        1. 0004-Added-in-the-hdfs-1520-1555-1554-from-tip-of-the-bra.patch
          40 kB
          Michael Stack
        2. 3285-v2.txt
          2 kB
          Michael Stack
        3. 3285-v3.txt
          2 kB
          Michael Stack
        4. hbaseRecoverHLog.patch
          2 kB
          Hairong Kuang
        5. hdfs-1520,1555,1554-for-cdh3b2.txt
          37 kB
          Michael Stack

        Issue Links

          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. HDFS-1520 HDFS 20 append: Lightweight NameNode operation to trigger lease recovery

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HBASE-3379 Log splitting slowed by repeated attempts at connecting to downed datanode

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              hairong Hairong Kuang
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: