Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9642

Refactor the core level snapshot cleanup mechanism to rely on Lucene

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.2
    • Fix Version/s: 6.3
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      SOLR-9269 introduced a mechanism to create/delete snapshots for a Solr core (using Lucene IndexDeletionPolicy). The current snapshot cleanup mechanism is based on reference counting the index files shared between multiple segments. Since this mechanism completely skips the Lucene APIs, it is not portable (e.g. it doesn't work on 4.10.3 version).

      I propose an alternative implementation which relies exclusively on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user hgadre opened a pull request:

          https://github.com/apache/lucene-solr/pull/97

          SOLR-9642 Refactor the snapshot cleanup mechanism to rely on Lucene

          The current snapshot cleanup mechanism is based on reference counting
          the index files shared between multiple segments. Since this mechanism
          completely skips the Lucene APIs, it is not portable (e.g. it doesn't
          work on 4.10.x version).

          This patch provides an alternate implementation which relies exclusively
          on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hgadre/lucene-solr SOLR-9642_fix

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/lucene-solr/pull/97.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #97


          commit 9f2eee48b9171eddcbb63aa716e8943cafeeddaf
          Author: Hrishikesh Gadre <hgadre@cloudera.com>
          Date: 2016-08-10T23:59:31Z

          SOLR-9642 Refactor the snapshot cleanup mechanism to rely on Lucene

          The current snapshot cleanup mechanism is based on reference counting
          the index files shared between multiple segments. Since this mechanism
          completely skips the Lucene APIs, it is not portable (e.g. it doesn't
          work on 4.10.x version).

          This patch provides an alternate implementation which relies exclusively
          on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user hgadre opened a pull request: https://github.com/apache/lucene-solr/pull/97 SOLR-9642 Refactor the snapshot cleanup mechanism to rely on Lucene The current snapshot cleanup mechanism is based on reference counting the index files shared between multiple segments. Since this mechanism completely skips the Lucene APIs, it is not portable (e.g. it doesn't work on 4.10.x version). This patch provides an alternate implementation which relies exclusively on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hgadre/lucene-solr SOLR-9642 _fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/97.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #97 commit 9f2eee48b9171eddcbb63aa716e8943cafeeddaf Author: Hrishikesh Gadre <hgadre@cloudera.com> Date: 2016-08-10T23:59:31Z SOLR-9642 Refactor the snapshot cleanup mechanism to rely on Lucene The current snapshot cleanup mechanism is based on reference counting the index files shared between multiple segments. Since this mechanism completely skips the Lucene APIs, it is not portable (e.g. it doesn't work on 4.10.x version). This patch provides an alternate implementation which relies exclusively on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hgadre commented on the issue:

          https://github.com/apache/lucene-solr/pull/97

          @yonik Could you please take a look?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hgadre commented on the issue: https://github.com/apache/lucene-solr/pull/97 @yonik Could you please take a look?
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          Looks good to me. I'll commit after some testing...

          Show
          yseeley@gmail.com Yonik Seeley added a comment - Looks good to me. I'll commit after some testing...
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 06ae3eb2ec71f97b39a4ab0b7579cda0397d9a1b in lucene-solr's branch refs/heads/branch_6x from Hrishikesh Gadre
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=06ae3eb ]

          SOLR-9642: Refactor the snapshot cleanup mechanism to rely on Lucene

          The current snapshot cleanup mechanism is based on reference counting
          the index files shared between multiple segments. Since this mechanism
          completely skips the Lucene APIs, it is not portable (e.g. it doesn't
          work on 4.10.x version).

          This patch provides an alternate implementation which relies exclusively
          on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup.

          mend

          Show
          jira-bot ASF subversion and git services added a comment - Commit 06ae3eb2ec71f97b39a4ab0b7579cda0397d9a1b in lucene-solr's branch refs/heads/branch_6x from Hrishikesh Gadre [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=06ae3eb ] SOLR-9642 : Refactor the snapshot cleanup mechanism to rely on Lucene The current snapshot cleanup mechanism is based on reference counting the index files shared between multiple segments. Since this mechanism completely skips the Lucene APIs, it is not portable (e.g. it doesn't work on 4.10.x version). This patch provides an alternate implementation which relies exclusively on Lucene IndexWriter (+ IndexDeletionPolicy) for cleanup. mend
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          Committed. Thanks!

          Show
          yseeley@gmail.com Yonik Seeley added a comment - Committed. Thanks!
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Yonik Seeley Thanks for taking a look! Shouldn't this be committed to trunk as well?

          Show
          hgadre Hrishikesh Gadre added a comment - Yonik Seeley Thanks for taking a look! Shouldn't this be committed to trunk as well?
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -
          Show
          yseeley@gmail.com Yonik Seeley added a comment - It was... it just hasn't shown up in this JIRA for some reason. https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=commit;h=46aeb52588c6ecf298ee894a79fb162e4e3437fe
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Yonik Seeley Ok cool!

          Show
          hgadre Hrishikesh Gadre added a comment - Yonik Seeley Ok cool!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hgadre closed the pull request at:

          https://github.com/apache/lucene-solr/pull/97

          Show
          githubbot ASF GitHub Bot added a comment - Github user hgadre closed the pull request at: https://github.com/apache/lucene-solr/pull/97
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Closing after 6.3.0 release.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Closing after 6.3.0 release.

            People

            • Assignee:
              yseeley@gmail.com Yonik Seeley
              Reporter:
              hgadre Hrishikesh Gadre
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development