Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12085

IndexFetcher does not honor SolrDeletionPolicy

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 5.4.1, 7.1, 7.2
    • Fix Version/s: None
    • Component/s: replication (java)
    • Labels:
      None

      Description

      Index Fetcher is not in sync with Solr Deletion Policy.

      If we set the SolrDeletionPolicy to retain more than 1 commit - i.e. maxCommitsToKeep > 1. Then slaves get stuck in alternate full replication. This happens because within IndexFetcher, there is a check for unusedFiles - hasUnusedFile, which takes IndexDirectory and latest commit as parameters. If there are files unrelated to latest commit in the index dir then this method returns true, and SolrDeletionPolicy is invoked. However there are no pending files to delete according to the index deletion policy since that is aware of other valid commits, and no action is taken. Once all the retries are exhausted, index fetcher sets fullreplication to true.  

      Possible solutions: 

      1. remove the check for hasUnusedFile completely. We invoke IndexWriter#deleteUnusedFiles and move on.
      2. Add another method in IndexFileDeleter (Lucene) which returns number of pending deleted files. We can replace the IndexFetcher#hasUnusedFile method with this.
      3. Keep track of unused files in IndexDeletionPolicyWrapper.    

      A variation of this bug was previously noted in https://issues.apache.org/jira/browse/SOLR-9278    

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Karishma Karishma Agrawal
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: