[HBASE-23812] [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.3.0
• Component/s: flakies
• Hadoop Flags: Reviewed
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 07/Feb/20 22:26
• Updated: 07/Apr/20 21:38
• Resolved: 10/Feb/20 17:46

Description

I've seen this fail. Here is the latest:

 [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 21.044 s <<< FAILURE! - in org.apache.hadoop.hbase.replication.regionserver.TestReplicator
 [ERROR] org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors  Time elapsed: 7.139 s  <<< FAILURE!
 java.lang.AssertionError: We did not replicate enough rows expected:<10> but was:<7>
   at org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)

In test logs there are a few (in groups of three) of these:

 4247 2020-02-07 13:50:29,070 WARN  [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors] regionserver.                       ReplicationSourceShipper(223): org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest threw unknown exception:
 4248 java.lang.ClassCastException: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be cast to java.io.IOException
 4249   at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
 4250   at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
 4251   at org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
 4252   at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
 4253   at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)

Let me fix this and see if it makes TestReplicator stable again.