Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-2200

DIH DocBuilder - Improve perf. on large delta deletes

    XMLWordPrintableJSON

Details

    Description

      In collectDelta, the procedure that collects the PKs for the documents that should be updated or deleted for an entity, iterates over the entire deltaSet for every deleted document. This is very expensive when you are updating and deleting millions of documents in one delta-import.
      Considering that the comparison between deleted and delta is on the PK, lets build the deltaSet as a HashMap instead of a HashSet to enable quick key lookups and remove the need for repeated iterations.

      Attachments

        1. SOLR-2200.patch
          3 kB
          Mark Waddle

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-1927 DocBuilder Inefficiency

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Activity

            People

              rcmuir Robert Muir
              markwaddle Mark Waddle
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: