Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9036

Solr slave is doing full replication (entire index) of index after master restart

    Details

      Description

      This was first described in the following email:
      https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201604.mbox/%3CCAFgNfOYN+xmpXwZwbjUzDDEUz7TJqHqktek6q7u8xgStqY3hRQ@mail.gmail.com%3E

      I tried Solr 5.3.1 and Solr 6 and I can reproduce the problem. If the master comes back online before the next polling interval then the slave finds itself in sync with the master but if the master is down for at least one polling interval then the slave pulls the entire full index from the master even if the index has not changed on the master.

      1. SOLR-9036.patch
        5 kB
        Shalin Shekhar Mangar
      2. SOLR-9036.patch
        5 kB
        Shalin Shekhar Mangar
      3. SOLR-9036.patch
        1.0 kB
        Shalin Shekhar Mangar

        Issue Links

          Activity

          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          This was caused by SOLR-7134 where any previous replication failure forces a full replication regardless of whether we are in SolrCloud mode or the cause of the last failure.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - This was caused by SOLR-7134 where any previous replication failure forces a full replication regardless of whether we are in SolrCloud mode or the cause of the last failure.
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Trivial patch to track replication success only in SolrCloud mode.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Trivial patch to track replication success only in SolrCloud mode.
          Hide
          markrmiller@gmail.com Mark Miller added a comment -

          Might be nice to add a little test, but +1 on patch.

          Show
          markrmiller@gmail.com Mark Miller added a comment - Might be nice to add a little test, but +1 on patch.
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Patch with a test that asserts that the slave downloads the index successfully only once

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Patch with a test that asserts that the slave downloads the index successfully only once
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Last patch had a comment in the test which was copied over from another test and made no sense. This patch removes it. I think this is ready.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Last patch had a comment in the test which was copied over from another test and made no sense. This patch removes it. I think this is ready.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 51b131950de0357fc64e0e951b887eb30a704cd1 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=51b1319 ]

          SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart

          Show
          jira-bot ASF subversion and git services added a comment - Commit 51b131950de0357fc64e0e951b887eb30a704cd1 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=51b1319 ] SOLR-9036 : Solr slave is doing full replication (entire index) of index after master restart
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 855d57bf10cc69037135242dd9911579b2046e84 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=855d57b ]

          SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart
          (cherry picked from commit 51b1319)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 855d57bf10cc69037135242dd9911579b2046e84 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=855d57b ] SOLR-9036 : Solr slave is doing full replication (entire index) of index after master restart (cherry picked from commit 51b1319)
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Thanks Mark!

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Thanks Mark!
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Re-opening because the new test added here fails often on slow machines.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Re-opening because the new test added here fails often on slow machines.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1dd877545fad0eae7be43fec109bceb4617fb6a4 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1dd8775 ]

          SOLR-9036: Disable doTestIndexFetchOnMasterRestart

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1dd877545fad0eae7be43fec109bceb4617fb6a4 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1dd8775 ] SOLR-9036 : Disable doTestIndexFetchOnMasterRestart
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit a6f9c8e171b8f48d5ced9c74b41f875aef567634 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6f9c8e ]

          SOLR-9036: Disable doTestIndexFetchOnMasterRestart
          (cherry picked from commit 1dd8775)

          Show
          jira-bot ASF subversion and git services added a comment - Commit a6f9c8e171b8f48d5ced9c74b41f875aef567634 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6f9c8e ] SOLR-9036 : Disable doTestIndexFetchOnMasterRestart (cherry picked from commit 1dd8775)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 93a33e0892e2ad3c8f73755d4ba10da72ff9c11a in lucene-solr's branch refs/heads/branch_6_0 from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=93a33e0 ]

          SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart
          (cherry picked from commit 51b1319)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 93a33e0892e2ad3c8f73755d4ba10da72ff9c11a in lucene-solr's branch refs/heads/branch_6_0 from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=93a33e0 ] SOLR-9036 : Solr slave is doing full replication (entire index) of index after master restart (cherry picked from commit 51b1319)
          Hide
          steve_rowe Steve Rowe added a comment -

          Backporting to 5.6 and 5.5.2.

          Show
          steve_rowe Steve Rowe added a comment - Backporting to 5.6 and 5.5.2.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 669b9cd00378cf14279ba81563b33807d3b68624 in lucene-solr's branch refs/heads/branch_5_5 from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=669b9cd ]

          SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart
          (cherry picked from commit 51b1319)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 669b9cd00378cf14279ba81563b33807d3b68624 in lucene-solr's branch refs/heads/branch_5_5 from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=669b9cd ] SOLR-9036 : Solr slave is doing full replication (entire index) of index after master restart (cherry picked from commit 51b1319)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 912c3da3eb7fa6d84b38357cd77c9727a24ef598 in lucene-solr's branch refs/heads/branch_5x from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=912c3da ]

          SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart
          (cherry picked from commit 51b1319)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 912c3da3eb7fa6d84b38357cd77c9727a24ef598 in lucene-solr's branch refs/heads/branch_5x from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=912c3da ] SOLR-9036 : Solr slave is doing full replication (entire index) of index after master restart (cherry picked from commit 51b1319)
          Hide
          arafalov Alexandre Rafalovitch added a comment -

          The work here seems to be all done. Safe to close?

          Show
          arafalov Alexandre Rafalovitch added a comment - The work here seems to be all done. Safe to close?
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          The fix has been released. I'll open another issue to fix the test which is currently disabled.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - The fix has been released. I'll open another issue to fix the test which is currently disabled.
          Hide
          praste Pushkar Raste added a comment - - edited

          Does fix for SOLR-9446 helps in this situation as well?

          Show
          praste Pushkar Raste added a comment - - edited Does fix for SOLR-9446 helps in this situation as well?
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          I don't see how it would. This bug affected master-slave (non-solr cloud) replication only.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - I don't see how it would. This bug affected master-slave (non-solr cloud) replication only.

            People

            • Assignee:
              shalinmangar Shalin Shekhar Mangar
              Reporter:
              shalinmangar Shalin Shekhar Mangar
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development