HBase
  1. HBase
  2. HBASE-2958

When hbase.hlog.split.skip.errors is set to false, we fail the split but thats it

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      When hbase.hlog.split.skip.errors is set to false, if we encounter an error splitting, splitting stops and exception is let propagate up the stack. I see that its caught in the new MasterFileSystem class and logged, but thats it. It would seem processing continues BUT we've dropped the edits in the split. We need to do better (default is hbase.hlog.split.skip.errors set to false – i.e. skip errors but keep going).

        Issue Links

          Activity

          Hide
          Cosmin Lehene added a comment -

          stack can you double check if this this still the case?

          Based on doc

          1.6.5.3.1. Handling of Errors During Log Splitting
          If you set the hbase.hlog.split.skip.errors option to true, errors are treated as follows:
          Any error encountered during splitting will be logged.
          The problematic WAL log will be moved into the .corrupt directory under the hbase rootdir,
          Processing of the WAL will continue
          

          also
          in WALSplitter.getNextLogLine we throw CorruptedLogFileException if we're set to skip errors and then in

          in WALSplitter.splitLogFile we mark the log as corrupted accordingly

              } catch (CorruptedLogFileException e) {
                LOG.warn("Could not parse, corrupted log file " + logPath, e);
                csm.getSplitLogWorkerCoordination().markCorrupted(rootDir,
                  logfile.getPath().getName(), fs);
                isCorrupted = true;
          
          Show
          Cosmin Lehene added a comment - stack can you double check if this this still the case? Based on doc 1.6.5.3.1. Handling of Errors During Log Splitting If you set the hbase.hlog.split.skip.errors option to true, errors are treated as follows: Any error encountered during splitting will be logged. The problematic WAL log will be moved into the .corrupt directory under the hbase rootdir, Processing of the WAL will continue also in WALSplitter.getNextLogLine we throw CorruptedLogFileException if we're set to skip errors and then in in WALSplitter.splitLogFile we mark the log as corrupted accordingly } catch (CorruptedLogFileException e) { LOG.warn( "Could not parse, corrupted log file " + logPath, e); csm.getSplitLogWorkerCoordination().markCorrupted(rootDir, logfile.getPath().getName(), fs); isCorrupted = true ;

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development