[HBASE-21043] TestShell list_procedures is flakey

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.0.1
• Component/s: shell, test
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 14/Aug/18 02:51
• Updated: 21/Aug/18 00:39
• Resolved: 20/Aug/18 20:28

Description

Fails 30% of the time in list_procedures. Fails creating a Procedure then trying to capture shell output to confirm it sees the just-queued Procedure only it looks like the Procedure finishes too quickly. It works for a while then there are a spate of failures. Then it works again.

Here is how it looks in test output:

Took 5.6355 secondsTook 0.0561 seconds...........F
===============================================================================
Failure: test_list_procedures(Hbase::ListProceduresTest)
src/test/ruby/shell/list_procedures_test.rb:65:in `block in test_list_procedures'
     62:         end
     63:       end
     64: 
  => 65:       assert_equal(1, matching_lines)
     66:     end
     67:   end
     68: end
<1> expected but was
<0>

Then in the log output for the test, I see this for the running of the Procedure:

2018-08-14 00:42:50,381 DEBUG [Time-limited test] procedure2.ProcedureExecutor(948): Stored pid=27, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.client.procedure.ShellTestProcedure
2018-08-14 00:42:50,397 INFO  [RS-EventLoopGroup-1-10] ipc.ServerRpcConnection(556): Connection from 67.195.81.150:50597, version=2.0.2-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService
F
===============================================================================
Failure: test_list_procedures(Hbase::ListProceduresTest)
src/test/ruby/shell/list_procedures_test.rb:65:in `block in test_list_procedures'
2018-08-14 00:42:50,586 INFO  [PEWorker-16] procedure2.ProcedureExecutor(1316): Finished pid=27, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.client.procedure.ShellTestProcedure in 234msec
     62:         end
     63:       end
     64: 
  => 65:       assert_equal(1, matching_lines)
     66:     end
     67:   end
     68: end
<1> expected but was
<0>
===============================================================================

The Procedure runs successfully but the regex test on the other end of the Admin is not finding what it expects – the Procedure ran in 234ms.

Will disable in a subprocedure for now till someone has time to play w/ this.

Issue Links

• is duplicated by: HBASE-20968 list_procedures_test fails due to no matching regex