Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1994

Master will lose hlog entries while splitting if region has empty oldlogfile.log

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.90.0
    • Fix Version/s: 0.20.3, 0.90.0
    • Component/s: master
    • Labels:
      None

      Description

      I don't know yet how an empty oldlogfile.log can exist, however it happened.
      Master will fail to put the splits in the region oldlogfile.log if an empty oldlogfile.log already exists there.
      This is the master log after I artificially reproduced it by placing an empty oldlogfile.log in /hbase/.META./1028785192/oldlogfile.log and then killed the regionserver that was holding the .META. table

      2009-11-19 09:08:36,012 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Splitting 1 hlog(s) in hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773
      2009-11-19 09:08:36,012 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Splitting hlog 1 of 1: hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773/hlog.dat.1258637493128, length=0
      2009-11-19 09:08:36,019 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Adding queue for .META.,,1
      2009-11-19 09:08:36,037 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Pushed=795 entries from hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773/hlog.dat.1258637493128
      2009-11-19 09:08:36,038 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Thread got 795 to process
      2009-11-19 09:08:36,043 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: Old hlog file hdfs://b0:9000/hbase/.META./1028785192/oldlogfile.log already exists. Copying existing file to new file
      2009-11-19 09:08:36,079 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: Got while writing region .META.,,1 log java.io.EOFException
      2009-11-19 09:08:36,081 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: hlog file splitting completed in 70 millis for hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773

        Attachments

        1. HBASE-1994-EOFE.patch
          2 kB
          Lars George

          Issue Links

            Activity

              People

              • Assignee:
                larsgeorge Lars George
                Reporter:
                clehene Cosmin Lehene
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Remaining Estimate - 48h
                  48h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified