Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12057

CDCR does not replicate to Collections with TLOG Replicas

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 7.2
    • Fix Version/s: None
    • Component/s: CDCR
    • Labels:
      None

      Description

      We created a collection using TLOG replicas in our QA clouds.
      We have a locally hosted solrcloud with 2 nodes, all our collections have 2 shards. We use CDCR to replicate the collections from this environment to 2 data centers hosted in Google cloud. This seems to work fairly well for our collections with NRT replicas. However the new TLOG collection has problems.
       
      The google cloud solrclusters have 4 nodes each (3 separate Zookeepers). 2 shards per collection with 2 replicas per shard.
       
      We never see data show up in the cloud collections, but we do see tlog files show up on the cloud servers. I can see that all of the servers have cdcr started, buffers are disabled.
      The cdcr source configuration is:
       
      "requestHandler":{"/cdcr":{
            "name":"/cdcr",
            "class":"solr.CdcrRequestHandler",
            "replica":[
             

      {           "zkHost":"[xxx-mzk01.sial.com:2181|http://xxx-mzk01.sial.com:2181/],[xxx-mzk02.sial.com:2181|http://xxx-mzk02.sial.com:2181/],[xxx-mzk03.sial.com:2181/solr|http://xxx-mzk03.sial.com:2181/solr]",           "source":"b2b-catalog-material-180124T",           "target":"b2b-catalog-material-180124T"}

      ,
             

      {           "zkHost":"[yyyy-mzk01.sial.com:2181|http://yyyy-mzk01.sial.com:2181/],[yyyy-mzk02.sial.com:2181|http://yyyy-mzk02.sial.com:2181/],[yyyy-mzk03.sial.com:2181/solr|http://yyyy-mzk03.sial.com:2181/solr]",           "source":"b2b-catalog-material-180124T",           "target":"b2b-catalog-material-180124T"}

      ],
            "replicator":

      {         "threadPoolSize":4,         "schedule":500,         "batchSize":250}

      ,
            "updateLogSynchronizer":{"schedule":60000}}}}
       
      The target configurations in the 2 clouds are the same:
      "requestHandler":{"/cdcr":{ "name":"/cdcr", "class":"solr.CdcrRequestHandler", "buffer":

      {"defaultState":"disabled"}

      }} 
       
      All of our collections have a timestamp field, index_date. In the source collection all the records have a date of 2/28/2018 but the target collections have a latest date of 1/26/2018
       
      I don't see cdcr errors in the logs, but we use logstash to search them, and we're still perfecting that. 
       
      We have a number of similar collections that behave correctly. This is the only collection that is a TLOG collection. It appears that CDCR doesn't support TLOG collections.
       
      It looks like the data is getting to the target servers. I see tlog files with the right timestamps. Looking at the timestamps on the documents in the collection none of the data appears to have been loaded.In the solr.log I see lots of /cdcr messages  action=LASTPROCESSEDVERSION,  action=COLLECTIONCHECKPOINT, and  action=SHARDCHECKPOINT 
       
      no errors
       
      Target collections autoCommit is set to  60000 I tried sending a commit explicitly no difference. cdcr is uploading data, but no new data appears in the collection.
       

        Attachments

        1. SOLR-12057.patch
          11 kB
          Amrit Sarkar
        2. SOLR-12057.patch
          24 kB
          Amrit Sarkar
        3. SOLR-12057.patch
          24 kB
          Amrit Sarkar
        4. SOLR-12057.patch
          24 kB
          Amrit Sarkar
        5. SOLR-12057.patch
          20 kB
          Amrit Sarkar
        6. SOLR-12057.patch
          26 kB
          Amrit Sarkar
        7. SOLR-12057.patch
          27 kB
          Amrit Sarkar
        8. cdcr-fail-with-tlog-pull.patch
          20 kB
          Amrit Sarkar
        9. cdcr-fail-with-tlog-pull.patch
          14 kB
          Amrit Sarkar

          Activity

            People

            • Assignee:
              varun Varun Thacker
              Reporter:
              WebHomer Webster Homer
            • Votes:
              4 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: