[HBASE-14411] Fix unit test failures when using multiwal as default WAL provider

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Yu Li
• Reporter: Yu Li
• Created: 11/Sep/15 18:58
• Updated: 01/Jul/22 21:26
• Resolved: 16/Sep/15 14:10

Description

If we set hbase.wal.provider to multiwal in hbase-server/src/test/resources/hbase-site.xml which allows us to use BoundedRegionGroupingProvider in UT, we will observe below failures in current code base:

Failed tests:
  TestHLogRecordReader>TestWALRecordReader.testPartialRead:164 expected:<1> but was:<2>
  TestHLogRecordReader>TestWALRecordReader.testWALRecordReader:216 expected:<2> but was:<3>
  TestWALRecordReader.testPartialRead:164 expected:<1> but was:<2>
  TestWALRecordReader.testWALRecordReader:216 expected:<2> but was:<3>
  TestDistributedLogSplitting.testRecoveredEdits:276 edits dir should have more than a single file in it. instead has 1
  TestAtomicOperation.testMultiRowMutationMultiThreads:499 expected:<0> but was:<1>
  TestHRegionServerBulkLoad.testAtomicBulkLoad:307
Expected: is <true>
     but: was <false>
  TestLogRolling.testCompactionRecordDoesntBlockRolling:611 Should have WAL; one table is not flushed expected:<1> but was:<0>
  TestLogRolling.testLogRollOnDatanodeDeath:359 null
  TestLogRolling.testLogRollOnPipelineRestart:472 Missing datanode should've triggered a log roll
  TestReplicationSourceManager.testLogRoll:237 expected:<6> but was:<7>
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestReplicationWALReaderManager.test:155 null
  TestWALSplit.testCorruptedLogFilesSkipErrorsFalseDoesNotTouchLogs:594 if skip.errors is false all files should remain in place expected:<11> but was:<12>
  TestWALSplit.testLogsGetArchivedAfterSplit:649 wrong number of files in the archive log expected:<11> but was:<12>
  TestWALSplit.testMovedWALDuringRecovery:810->retryOverHdfsProblem:793 expected:<11> but was:<12>
  TestWALSplit.testRetryOpenDuringRecovery:838->retryOverHdfsProblem:793 expected:<11> but was:<12>
  TestWALSplitCompressed>TestWALSplit.testCorruptedLogFilesSkipErrorsFalseDoesNotTouchLogs:594 if skip.errors is false all files should remain in place expected:<11> but was:<12>
  TestWALSplitCompressed>TestWALSplit.testLogsGetArchivedAfterSplit:649 wrong number of files in the archive log expected:<11> but was:<12>
  TestWALSplitCompressed>TestWALSplit.testMovedWALDuringRecovery:810->TestWALSplit.retryOverHdfsProblem:793 expected:<11> but was:<12>
  TestWALSplitCompressed>TestWALSplit.testRetryOpenDuringRecovery:838->TestWALSplit.retryOverHdfsProblem:793 expected:<11> but was:<12>

While patch for HBASE-14306 could resolve failures of TestHLogRecordReader, TestReplicationSourceManager and TestReplicationWALReaderManager, this JIRA will focus on resolving the others

Issue Links

• Is contained by: HBASE-14457 Umbrella: Improve Multiple WAL for production usage