Solr
  1. Solr
  2. SOLR-6679

disable/remove suggester from stock solrconfig

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.10
    • Fix Version/s: 4.10.3, 5.0
    • Component/s: None
    • Labels:
      None

      Description

      The stock solrconfig provides a bad experience with a large index... start up Solr and it will spin at 100% CPU for minutes, unresponsive, while it apparently builds a suggester index.

        Issue Links

          Activity

          Hide
          Yonik Seeley added a comment -

          This is what I did:
          1) indexed 10M very small docs (only takes a few minutes).
          2) shut down Solr
          3) start up Solr and watch it be unresponsive for over 4 minutes!

          I didn't even use any of the fields specified in the suggester config and I never called the suggest request handler.
          If this can't be made truly lazy, then it should just be removed/disabled for the stock solrconfig.
          I've already made this change for Heliosearch: https://github.com/Heliosearch/heliosearch/commit/b43988585f948965a5760bf60314fc76b1c099ff

          Show
          Yonik Seeley added a comment - This is what I did: 1) indexed 10M very small docs (only takes a few minutes). 2) shut down Solr 3) start up Solr and watch it be unresponsive for over 4 minutes! I didn't even use any of the fields specified in the suggester config and I never called the suggest request handler. If this can't be made truly lazy, then it should just be removed/disabled for the stock solrconfig. I've already made this change for Heliosearch: https://github.com/Heliosearch/heliosearch/commit/b43988585f948965a5760bf60314fc76b1c099ff
          Hide
          Jan Høydahl added a comment -

          +1
          Background thread build?
          Would also like something more flexible than pure buildOnCommit. Perhaps an additional buildMinInterval=<seconds> could tell the component that on a commit, it should only re-build if time-since-last-build > buildMinInterval.

          Show
          Jan Høydahl added a comment - +1 Background thread build? Would also like something more flexible than pure buildOnCommit. Perhaps an additional buildMinInterval=<seconds> could tell the component that on a commit, it should only re-build if time-since-last-build > buildMinInterval.
          Hide
          Yonik Seeley added a comment -

          Give that I noticed 2 people who were bit by this in the last week (and who knows how many people that haven't noticed), we should probably comment out the suggester for the upcoming 4.10.3 release as well.

          Show
          Yonik Seeley added a comment - Give that I noticed 2 people who were bit by this in the last week (and who knows how many people that haven't noticed), we should probably comment out the suggester for the upcoming 4.10.3 release as well.
          Hide
          Shalin Shekhar Mangar added a comment -

          +1

          Show
          Shalin Shekhar Mangar added a comment - +1
          Hide
          Mohsen added a comment -

          +1
          If you add reload() and build() to the subject of this issue more people will vote for this.

          Show
          Mohsen added a comment - +1 If you add reload() and build() to the subject of this issue more people will vote for this.
          Hide
          ASF subversion and git services added a comment -

          Commit 1638010 from Yonik Seeley in branch 'dev/trunk'
          [ https://svn.apache.org/r1638010 ]

          SOLR-6679: comment out suggester in stock config due to long startup times

          Show
          ASF subversion and git services added a comment - Commit 1638010 from Yonik Seeley in branch 'dev/trunk' [ https://svn.apache.org/r1638010 ] SOLR-6679 : comment out suggester in stock config due to long startup times
          Hide
          ASF subversion and git services added a comment -

          Commit 1638012 from Yonik Seeley in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1638012 ]

          SOLR-6679: comment out suggester in stock config due to long startup times

          Show
          ASF subversion and git services added a comment - Commit 1638012 from Yonik Seeley in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1638012 ] SOLR-6679 : comment out suggester in stock config due to long startup times
          Hide
          ASF subversion and git services added a comment -

          Commit 1638013 from Yonik Seeley in branch 'dev/branches/lucene_solr_4_10'
          [ https://svn.apache.org/r1638013 ]

          SOLR-6679: comment out suggester in stock config due to long startup times

          Show
          ASF subversion and git services added a comment - Commit 1638013 from Yonik Seeley in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1638013 ] SOLR-6679 : comment out suggester in stock config due to long startup times
          Hide
          Hoss Man added a comment -

          rather then completley comment this out, i'd like to propose that we add this back in but have it completley disabled by default and only usable if a sysprop is set on startup. (similar to how result clustering works)

          if nothing else, this will make it easier to get hte docs updated, and let people try the feature out w/o needing to manually hack at the configs

          Attached patch shows what i mean.

          Show
          Hoss Man added a comment - rather then completley comment this out, i'd like to propose that we add this back in but have it completley disabled by default and only usable if a sysprop is set on startup. (similar to how result clustering works) if nothing else, this will make it easier to get hte docs updated, and let people try the feature out w/o needing to manually hack at the configs Attached patch shows what i mean.
          Hide
          Hoss Man added a comment -

          updating summary to clarify focus of issue on disabling/removing (since SOLR-6845 now exists to focus on fixing)

          Show
          Hoss Man added a comment - updating summary to clarify focus of issue on disabling/removing (since SOLR-6845 now exists to focus on fixing)
          Hide
          Yonik Seeley added a comment -

          Yep, as long as it works (to avoid impact if not used).
          +1

          Show
          Yonik Seeley added a comment - Yep, as long as it works (to avoid impact if not used). +1
          Hide
          ASF subversion and git services added a comment -

          Commit 1645721 from hossman@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1645721 ]

          SOLR-6679: uncomment /suggest, but tie it to an sysprop so you have to go out of your way to enable on startup

          Show
          ASF subversion and git services added a comment - Commit 1645721 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1645721 ] SOLR-6679 : uncomment /suggest, but tie it to an sysprop so you have to go out of your way to enable on startup
          Hide
          ASF subversion and git services added a comment -

          Commit 1645722 from hossman@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1645722 ]

          SOLR-6679: uncomment /suggest, but tie it to an sysprop so you have to go out of your way to enable on startup (merge r1645721)

          Show
          ASF subversion and git services added a comment - Commit 1645722 from hossman@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1645722 ] SOLR-6679 : uncomment /suggest, but tie it to an sysprop so you have to go out of your way to enable on startup (merge r1645721)
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Unassigned
              Reporter:
              Yonik Seeley
            • Votes:
              2 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development