Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7820

IndexFetcher should calculate ahead of time how much space is needed for full snapshot based recovery and cleanly abort instead of trying and running out of space on a node

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: replication (java)
    • Labels:
      None

      Description

      When a replica is trying to recover and it's IndexFetcher decides it needs to pull the full index from a peer (isFullCopyNeeded == true), then the existing index directory should be deleted before the full copy is started to free up disk to pull a fresh index, otherwise the server will potentially need 2x the disk space (old + incoming new). Currently, the IndexFetcher removes the index directory after the new is downloaded; however, once the fetcher decides a full copy is needed, what is the value of the existing index? It's clearly out-of-date and should not serve queries. Since we're deleting data preemptively, maybe this should be an advanced configuration property, only to be used by those that are disk-space constrained (which I'm seeing more and more with people deploying high-end SSDs - they typically don't have 2x the disk capacity required by an index).

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              thelabdude Timothy Potter
            • Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: