Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9908

create SolrCloudDIHWriter to speedup DataImportHandler on SolrCloud

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Right now if DIH is invoked in SolrCloud it feeds docs one by one synchronously via DistributedUpdateProcessor.
      It's proposed to create DIHWriter implementation which will stream docs with SolrCloudClient. I expect per-shard parallelism and even more with CloudSolrClient.setParallelUpdates(true).
      What's your feeling about it?

        Attachments

        1. DIH-SolrCloud-test.patch
          21 kB
          Mikhail Khludnev

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mkhl Mikhail Khludnev
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: