Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-766

FileNotFoundException trying to load HStoreFile 'data'

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.2.0
    • None
    • None

    Description

      From Renaud Delbru up on the list:

      Hi,
      
      after our issues ("Replay of HLog required", in a precious thread) with HBase, it seems that HBase has corrupted regions.
      We have, on the three region servers, errors stating that HBase cannot open certain regions because some map files on hdfs are missing (see the log attached).
      
      Do you have any ideas how to fix this ?
      
      Thanks.
      
      java.io.FileNotFoundException: File does not exist: hdfs://hadoop1.sindice.net:54310/hbase/page-repository/1105668475/field/mapfiles/5122893264992435570/data
        at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:369)
        at org.apache.hadoop.hbase.regionserver.HStoreFile.length(HStoreFile.java:464)
        at org.apache.hadoop.hbase.regionserver.HStore.loadHStoreFiles(HStore.java:409)
        at org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:236)
        at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1575)
        at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:451)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:901)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:876)
        at org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:816)
        at java.lang.Thread.run(Thread.java:619)
      
      

      I'd thought we'd added handling of this kind of event post HDFS crashings but looking in code, it seems that TRUNK may not have been fixed up properly.

      Attachments

        1. hs.patch
          0.7 kB
          Michael Stack
        2. 766.patch
          0.7 kB
          Michael Stack

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stack Michael Stack
            stack Michael Stack
            Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment