Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11278

Fix race in cdcr bootstrap process

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 6.6.1, 7.0
    • 7.1
    • CDCR

    Description

      CdcrBootstrapTest is failing while running beasts for significant iterations.

      The bootstrapping is failing in the test, after the first batch is indexed for each testmethod, which results in documents mismatch ::

        [beaster]   2> 39167 ERROR (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2) [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation failed
        [beaster]   2> java.util.concurrent.ExecutionException: java.lang.AssertionError
        [beaster]   2> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        [beaster]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
        [beaster]   2> 	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
        [beaster]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        [beaster]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        [beaster]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        [beaster]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        [beaster]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        [beaster]   2> 	at java.lang.Thread.run(Thread.java:748)
        [beaster]   2> Caused by: java.lang.AssertionError
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
        [beaster]   2> 	at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
        [beaster]   2> 	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        [beaster]   2> 	... 5 more
      
        [beaster] [01:37:16.282] FAILURE  153s | CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
        [beaster]    > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1000>
      

      Attachments

        1. master-bs.patch
          11 kB
          Amrit Sarkar
        2. SOLR-11278.patch
          3 kB
          Amrit Sarkar
        3. SOLR-11278.patch
          2 kB
          Amrit Sarkar
        4. SOLR-11278.patch
          3 kB
          Amrit Sarkar
        5. SOLR-11278.patch
          7 kB
          Amrit Sarkar
        6. SOLR-11278-awaits-fix.patch
          0.7 kB
          Amrit Sarkar
        7. SOLR-11278-cancel-bootstrap-on-stop.patch
          2 kB
          Shalin Shekhar Mangar
        8. test_results
          14.40 MB
          Amrit Sarkar

        Activity

          People

            varun Varun Thacker
            sarkaramrit2@gmail.com Amrit Sarkar
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: