Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10014

Log a warning when the number of fields in a core exceeds a configurable value

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 4.10.4
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      When the number of fields in an index gets extremely large, major performance problems can occur. If the number of fields in a core exceeds a configurable number, with a default somewhere around 10000, a warning should be logged when the SolrCore is first created. A decision needs to be made about whether to repeat the warning on core reload ... my instinct is that it should NOT be repeated, but I can see where a repeat might have some value. Logging on reloads as well as startup would likely be easier.

      This was discovered by a Solr user who had a 420MB index with 650K documents, but their applications were abusing dynamic fields to the point where they had about 2 million unique fields in the index. The small size of the index should have resulted in extremely fast commit times, but commits were taking about 10 seconds because of what Lucene had to do to handle all those fields.

        Attachments

        1. SOLR-10014.patch
          2 kB
          Shawn Heisey

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                elyograg Shawn Heisey
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: