[HBASE-8333] TestTableLockManager#testDelete may occasionally fail due to lack of synchronization between test and handler thread

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.98.0, 0.95.1
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 12/Apr/13 13:18
• Updated: 23/Sep/13 19:08
• Resolved: 13/Apr/13 20:56

Description

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertTrue(Assert.java:52)
	at org.apache.hadoop.hbase.master.TestTableLockManager.testDelete(TestTableLockManager.java:250)

It corresponds to this assertion:

    assertTrue(ZKUtil.checkExists(zkWatcher,
        ZKUtil.joinZNode(zkWatcher.tableLockZNode, Bytes.toString(TABLE_NAME))) < 0);

In test output (https://builds.apache.org/job/HBase-TRUNK/4056/testReport/junit/org.apache.hadoop.hbase.master/TestTableLockManager/testDelete/):

2013-04-12 10:46:02,642 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] handler.DeleteTableHandler(113): Table 'TestTableLevelLocks' archived!
2013-04-12 10:46:02,642 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] handler.DeleteTableHandler(117): Removing 'TestTableLevelLocks' descriptor.
2013-04-12 10:46:02,643 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] handler.DeleteTableHandler(121): Marking 'TestTableLevelLocks' as deleted.
2013-04-12 10:46:02,658 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] master.TableLockManager$ZKTableLockManager$TableLockImpl(253): Attempt to release table write lock on :TestTableLevelLocks
2013-04-12 10:46:02,667 DEBUG [RS_CLOSE_META-janus.apache.org,54489,1365763551867-0] util.FSUtils(211): Creating file=hdfs://localhost:52497/user/jenkins/hbase/.META./1028785192/.tmp/59ec68e4587d43d6bc026e4d6f2aafa2 with permission=rwxrwxrwx
2013-04-12 10:46:02,674 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] lock.ZKInterProcessLockBase(297): Successfully released /hbase/table-lock/TestTableLevelLocks/write-master:362650000000002
2013-04-12 10:46:02,675 DEBUG [MASTER_TABLE_OPERATIONS-janus.apache.org,36265,1365763551669-0] master.TableLockManager$ZKTableLockManager$TableLockImpl(269): Released table lock on :TestTableLevelLocks

This seems to be timing issue: when admin.deleteTable(TABLE_NAME) returned, TableLockManager hadn't got around to finishing the release of table lock.