Solr
  1. Solr
  2. SOLR-310

Deleting pending handling can cause OOM

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3
    • Component/s: None
    • Labels:
      None

      Description

      > Mike:
      >> Could happen when doDeleting the pending docs too. James: try
      >> sending commit every 500k docs or so.

      Yonik:
      > Hmmm, right... some of the memory usage will be related to the treemap
      > keeping track of deleted items, and some of it will be related to the
      > reader itself (the term index, and the norms).

      > Perhaps we should have some sort of limit on the number of deletions
      > we cache so people don't have to worry about that part.

      Good idea. Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?

        Activity

        Hide
        Mike Klaas added a comment -

        q&d patch that implements the above. Called it maxPendingDeletes, since the complexity/memory being bounded is determined by that, not pending docs.

        Show
        Mike Klaas added a comment - q&d patch that implements the above. Called it maxPendingDeletes, since the complexity/memory being bounded is determined by that, not pending docs.
        Hide
        Mike Klaas added a comment -

        committed in r572831

        Show
        Mike Klaas added a comment - committed in r572831

          People

          • Assignee:
            Mike Klaas
            Reporter:
            Mike Klaas
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development