[HBASE-11100] IntegrationTestTableSnapshotInputFormat fails due to empty regions

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.98.1
• Component/s: snapshots
• Assignee: Unassigned
• Reporter: Vandana Ayyalasomayajula
• Created: 30/Apr/14 16:50
• Updated: 17/Jun/22 05:38
• Resolved: 30/Apr/14 17:53

Description

When IntegrationTestTableSnapshotInputFormat is ran on real cluster with command

hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestTableSnapshotInputFormat

it fails with the following exception:

Exception in thread "main" java.lang.AssertionError: expected:<32> but was:<30>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:555)
	at org.junit.Assert.assertEquals(Assert.java:542)
	at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.confirmSnapshotValid(SnapshotTestingUtils.java:244)
	at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.createSnapshotAndValidate(SnapshotTestingUtils.java:399)
	at org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat.createTableAndSnapshot(TestTableSnapshotInputFormat.java:193)
	at org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat.doTestWithMapReduce(TestTableSnapshotInputFormat.java:363)
	at org.apache.hadoop.hbase.mapreduce.IntegrationTestTableSnapshotInputFormat.runTestFromCommandLine(IntegrationTestTableSnapshotInputFormat.java:132)
	at org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:79)
	at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.mapreduce.IntegrationTestTableSnapshotInputFormat.main(IntegrationTestTableSnapshotInputFormat.java:151)

However when we specify less number of regions, say 6 like

hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestTableSnapshotInputFormat -DIntegrationTestTableSnapshotInputFormat.numRegions=6

the test passes.

The integration test internally uses HBaseTestingUtility's loadtable API to create and load data into the regions. But that API does not seem to generate enough data for 32 regions ( which is the default number of regions in the integration test). As a result some regions end up with no keys. When the SnapshotTestingUtils calls confirmSnapshotValid method to verify the snaphot process, it looks for store files. When some region are empty they are not counted and hence the test fails.