[HBASE-18340] TestXXXProcedure tests hanging or failing on branch-1 (branch-1.4)

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Assignee: Unassigned
• Reporter: Andrew Kyle Purtell
• Created: 07/Jul/17 23:52
• Updated: 01/Aug/18 06:23
• Resolved: 11/Jul/17 02:26

Description

TestAddColumnFamilyProcedure - hanging
TestCreateNamespaceProcedure - failing
TestDeleteNamespaceProcedure - failing
TestDisableTableProcedure - hanging
TestModifyColumnFamilyProcedure - hanging
TestModifyTableProcedure - hanging
TestTruncateTableProcedure - hanging

Tests are still running so this is not a complete list of all of them, but I think they are all going to broken. Just tried TestEnableTableProcedure and that is hanging too.

Tests run: 7, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 6.63 sec <<< FAILURE! - in org.apache.hadoop.hbase.master.procedure.TestCreateNamespaceProcedure
testRollbackAndDoubleExecution(org.apache.hadoop.hbase.master.procedure.TestCreateNamespaceProcedure) Time elapsed: 2.053 sec <<< FAILURE!
java.lang.AssertionError: expected a running proc
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.assertProcNotYetCompleted(ProcedureTestingUtility.java:204)
at org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.testRollbackAndDoubleExecution(MasterProcedureTestingUtility.java:279)
at org.apache.hadoop.hbase.master.procedure.TestCreateNamespaceProcedure.testRollbackAndDoubleExecution(TestCreateNamespaceProcedure.java:219)

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 525.635 sec - in org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure
Running org.apache.hadoop.hbase.master.procedure.TestDeleteNamespaceProcedure
Tests run: 6, Failures: 2, Errors: 1, Skipped: 0, Time elapsed: 9.331 sec <<< FAILURE! - in org.apache.hadoop.hbase.master.procedure.TestDeleteNamespacePr
ocedure
testRollbackAndDoubleExecution(org.apache.hadoop.hbase.master.procedure.TestDeleteNamespaceProcedure) Time elapsed: 3.528 sec <<< FAILURE!
java.lang.AssertionError: expected a running proc
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility.assertProcNotYetCompleted(ProcedureTestingUtility.java:204)
at org.apache.hadoop.hbase.master.procedure.MasterProcedureTestingUtility.testRollbackAndDoubleExecution(MasterProcedureTestingUtility.java:279)
at org.apache.hadoop.hbase.master.procedure.TestDeleteNamespaceProcedure.testRollbackAndDoubleExecution(TestDeleteNamespaceProcedure.java:198)

Tests seem to be hanging in tearDown

"main" #1 prio=5 os_prio=31 tid=0x00007fcdbd806000 nid=0x1703 in Object.wait() [0x0000700000218000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:186)

• locked <0x0000000716820ae0> (a java.util.concurrent.atomic.AtomicBoolean)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4443)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4435)
  at org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsyncV2(HBaseAdmin.java:1370)
  at org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1329)
  at org.apache.hadoop.hbase.HBaseTestingUtility.deleteTable(HBaseTestingUtility.java:1966)
  at org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure.tearDown(TestDisableTableProcedure.java:79)