[SOLR-11467] CdcrBootstrapTest::testBootstrapWithContinousIndexingOnSourceCluster Failure

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 6.6, 6.6.1, 7.0, 7.0.1, 7.1
• Component/s: CDCR
• Assignee: Varun Thacker
• Reporter: Amrit Sarkar
• Created: 11/Oct/17 15:16
• Updated: 28/Oct/20 20:37
• Resolved: 28/Oct/20 20:37

Description

CdcrBootstrapTest is still failing in master and other branches with:

[junit4] FAILURE  130s J1 | CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1901>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([41753A7BCCA7C778:953071222BF17483]:0)
   [junit4]    > 	at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)

ref: https://jenkins.thetaphi.de/job/Lucene-Solr-7.0-Linux/423/

From one of the failed Solr jenkins log:

[junit4] 2> 1143166 INFO (cdcr-replicator-4421-thread-1-processing-n:127.0.0.1:62832_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:62832_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicator Forwarded 991 updates to target cdcr-target
[junit4] 2> 1144176 INFO (cdcr-replicator-4421-thread-1-processing-n:127.0.0.1:62832_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:62832_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicator Forwarded 909 updates to target cdcr-target
[junit4] 2> 1145118 INFO (cdcr-replicator-4421-thread-1-processing-n:127.0.0.1:62832_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:62832_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicator Forwarded 0 updates to target cdcr-target

Total 1900 updates were sent, instead of 2000. Ideally the bootstrap process is responsible for 1000, and normal cdc replication is responsble for 1000. On looking closely, the bootstrap is completed successfully. We are 100% sure here, bootstrap worked w/o any issue. And still 1900 updates were sent via replicator, instead of 1000.