Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11504

Provide a config to restrict number of indexing threads

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 5.3, 6.0, 7.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      For heavy indexing load (through REST api), Solr does not have any way to restrict number of threads. There used to be a config in lucene to restrict number of threads but that has been removed since https://issues.apache.org/jira/browse/LUCENE-6659 .

      For example, in my bulk indexing scenario, within few minutes, my solr server had created 300 parallel threads each writing its own segment. The result was tons of small segments getting flushed to disk (as total RAM limit was reached quickly by sum of all segments), and solr has to spend time later to merge them into reasonable sizes.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                niqbal Nawab Zada Asad iqbal
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 336h
                  336h
                  Remaining:
                  Remaining Estimate - 336h
                  336h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified