Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11326

CDCR bootstrap should not download tlog's from source

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.2
    • Component/s: None
    • Labels:
      None

      Description

      While analyzing two separate fails on SOLR-11278 I see that during bootstrap the tlog's from the source is getting download

      snippet1:

         [junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
         [junit4]   2> 42934 INFO  (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
         [junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
         [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
         [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 1503514968639
         [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
         [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
         [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication process
         [junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, name=tlog.0000000000000000000.1576549701811961856}, {size=4770, name=tlog.0000000000000000001.1576549702515556352}, {size=4770, name=tlog.0000000000000000002.1576549702628802560}, {size=4770, name=tlog.0000000000000000003.1576549702720028672}, {size=4770, name=tlog.0000000000000000004.1576549702799720448}, {size=4770, name=tlog.0000000000000000005.1576549702894092288}, {size=4770, name=tlog.0000000000000000006.1576549703029358592}, {size=4770, name=tlog.0000000000000000007.1576549703126876160}, {size=4770, name=tlog.0000000000000000008.1576549703208665088}, {size=4770, name=tlog.0000000000000000009.1576549703295696896}
      

      snippet2:

       17070    [junit4]   2> 677606 INFO  (qtp22544544-5725) [    ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1^M
       17071    [junit4]   2> 677608 INFO  (qtp22544544-5725) [    ] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor^M
      17091    [junit4]   2> 677627 INFO  (qtp22544544-5724) [    ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0^M
       17092    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 12^M
       17093    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 1504127155366^M
       17094    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1^M
       17095    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0^M
       17096    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Starting replication process^M
       17097    [junit4]   2> 677654 INFO  (qtp22544544-5725) [    ] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4773, name=tlog.0000000000000000009}]^M
       17098    [junit4]   2> 677655 INFO  (qtp22544544-5725) [    ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1]  webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=26^M
       17099    [junit4]   2> 677656 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Number of files in latest index in master: 23^M
       17100    [junit4]   2> 677656 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher Number of tlog files in master: 1^M
      

      Also here's the code comment which also says we don't need to download the tlog's

      CdcrRequestHandler.java
              ModifiableSolrParams solrParams = new ModifiableSolrParams();
              solrParams.set(ReplicationHandler.MASTER_URL, masterUrl);
              // we do not want the raw tlog files from the source
              solrParams.set(ReplicationHandler.TLOG_FILES, false);
      

        Attachments

        1. SOLR-11326.patch
          13 kB
          Amrit Sarkar
        2. SOLR-11326.patch
          13 kB
          Varun Thacker
        3. SOLR-11326.patch
          15 kB
          Varun Thacker
        4. WITHOUT-FIX.patch
          6 kB
          Amrit Sarkar
        5. SOLR-11326.patch
          8 kB
          Amrit Sarkar
        6. SOLR-11326.patch
          2 kB
          Amrit Sarkar
        7. SOLR-11326.patch
          2 kB
          Varun Thacker

          Activity

            People

            • Assignee:
              varunthacker Varun Thacker
              Reporter:
              varunthacker Varun Thacker
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: