Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9908

create SolrCloudDIHWriter to speedup DataImportHandler on SolrCloud

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      Right now if DIH is invoked in SolrCloud it feeds docs one by one synchronously via DistributedUpdateProcessor.
      It's proposed to create DIHWriter implementation which will stream docs with SolrCloudClient. I expect per-shard parallelism and even more with CloudSolrClient.setParallelUpdates(true).
      What's your feeling about it?

      Attachments

        1. DIH-SolrCloud-test.patch
          21 kB
          Mikhail Khludnev

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. SOLR-4058 DIH should use the SolrCloudServer impl when running in SolrCloud mode.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SOLR-3666 DataImportHandler status command in SolrCloud does not work properly

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-7188 Run Data Import Handler processes in a SolrJ client

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              mkhl Mikhail Khludnev
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: