Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12978

Region goes permanently offline (WAS: hbase:meta has a row missing hregioninfo and it causes my long-running job to fail)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 1.0.0, 1.1.0
    • None
    • None
    • Reviewed
    • Hide
      Fixes a bug where an optimization that keeps our hfile indexes files small chanced upon an entry that has special meaning in the meta table making it so we failed to find a region entry in an hbase:meta HFile (though the entry was present). Once we'd happened upon this condition, the region would be unfindable thereafter making it so the region was considered offline. The bug is in branch-1+ hbase only.
      Show
      Fixes a bug where an optimization that keeps our hfile indexes files small chanced upon an entry that has special meaning in the meta table making it so we failed to find a region entry in an hbase:meta HFile (though the entry was present). Once we'd happened upon this condition, the region would be unfindable thereafter making it so the region was considered offline. The bug is in branch-1+ hbase only.

    Description

      Testing 1.0.0 trying long-running tests.

      A row in hbase:meta was missing its HRI entry. It caused the job to fail. Around the time of the first task failure, there are balances of the hbase:meta region and it was on a server that crashed. I tried to look at what happened around time of our writing hbase:meta and I ran into another issue; 20 logs of 256MBs filled with WrongRegionException written over a minute or two. The actual update of hbase:meta was not in the logs, it'd been rotated off.

      Attachments

        1. 12978v2.txt
          9 kB
          Michael Stack
        2. 12978.txt
          4 kB
          Michael Stack
        3. e7cadabc6e5e46c7bf6b3d445f0c53cf
          725 kB
          Michael Stack

        Activity

          People

            stack Michael Stack
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: