Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11278

Fix race in cdcr bootstrap process

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 6.6.1, 7.0
    • Fix Version/s: 7.1
    • Component/s: CDCR
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:

      Description

      CdcrBootstrapTest is failing while running beasts for significant iterations.

      The bootstrapping is failing in the test, after the first batch is indexed for each testmethod, which results in documents mismatch ::

        [beaster]   2> 39167 ERROR (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2) [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation failed
        [beaster]   2> java.util.concurrent.ExecutionException: java.lang.AssertionError
        [beaster]   2> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        [beaster]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
        [beaster]   2> 	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
        [beaster]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        [beaster]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        [beaster]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        [beaster]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        [beaster]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        [beaster]   2> 	at java.lang.Thread.run(Thread.java:748)
        [beaster]   2> Caused by: java.lang.AssertionError
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
        [beaster]   2> 	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        [beaster]   2> 	... 5 more
      
        [beaster] [01:37:16.282] FAILURE  153s | CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
        [beaster]    > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1000>
      

        Attachments

        1. master-bs.patch
          11 kB
          Amrit Sarkar
        2. SOLR-11278.patch
          3 kB
          Amrit Sarkar
        3. SOLR-11278.patch
          2 kB
          Amrit Sarkar
        4. SOLR-11278.patch
          3 kB
          Amrit Sarkar
        5. SOLR-11278.patch
          7 kB
          Amrit Sarkar
        6. SOLR-11278-awaits-fix.patch
          0.7 kB
          Amrit Sarkar
        7. SOLR-11278-cancel-bootstrap-on-stop.patch
          2 kB
          Shalin Shekhar Mangar
        8. test_results
          14.40 MB
          Amrit Sarkar

          Activity

            People

            • Assignee:
              varunthacker Varun Thacker
              Reporter:
              sarkaramrit2@gmail.com Amrit Sarkar
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: