Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1342

Add to filesystem info needed to rebuild .META.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.20.0
    • None
    • None
    • Incompatible change, Reviewed

    Description

      From Cosmin up on list:

      Hi,
      
      Is there any way to rebuild the .META. Table with information in HDFS?
      
      Long story short, the namenode ran out of disk and crashed and we couldn't
      have it load the edits.new file at startup (the edits.new file seemed
      corrupted and it failed to read some Long value) so we removed the edits.new
      file. Apparently the edits file had 10 - 15 entries. Some edit data on
      -ROOT-, some edit data on .META., some data on a table compaction
      (comapction.dir) and some data on some hbase logs. Only last entry seem to
      be bogus, having almost a megabyte of trailing zeroes.
      
      After renaming edits.new HDFS started ok and HBase data is present. However
      when we started Hbase one table was missing entirely and one table was
      empty. The HBase master report shows Regions On FS: 94, but the report on
      region servers shows only 10 regions on all the region servers.
      
      Is there any way to cleanup hbase and rebuild the .META. table from HDFS?
      This would be ok even if it implies throwing up some data.
      
      If we would clean the edits.new file and restart HDFS or if we would
      manually apply the edits would that still make consistent changes to HBase
      data?
      
      Thanks,
      Cosmin
      

      Lets log the info needed to rebuild your .META. or add things like HRegionInfo made creating a region as a file under each region directory. That don't seem hard and the info could be used rebuilding. What else would we need?

      Marking as blocker/critical for 0.20.0.

      Attachments

        1. 1342.patch
          6 kB
          Michael Stack

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment