Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-4399

NPE in Lucene if Solr spellcheck sourceLocation parameter is missing for FileBasedSpellChecker or IndexBasedSpellChecker

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0
    • Fix Version/s: None
    • Component/s: spellchecker
    • Labels:
      None

      Description

      An NPE will occur down in Lucene SpellChecker if the sourceLocation parameter is missing for the FileBasedSpellChecker and the spellcheck.build parameter is specified on a query request. Solr should check if the sourceLocation parameter is missing and either give a readable exception explaining the problem, or use a reasonable default such as "spelling.txt".

      To repro, add this to the Solr 4.0 example solrconfig.xml:

        <requestHandler name="/spellBug4" class="solr.SearchHandler" startup="lazy">
          <lst name="defaults">
            <str name="df">name</str>
            <str name="q.op">AND</str>
            <str name="spellcheck">on</str>
            <str name="spellcheck.build">on</str>
          </lst>
          <arr name="last-components">
            <str>spellcheckBug4</str>
          </arr>
        </requestHandler>
      
        <searchComponent name="spellcheckBug4" class="solr.SpellCheckComponent">
          <lst name="spellchecker">
            <str name="classname">solr.FileBasedSpellChecker</str>
            <!-- MISSING: <str name="sourceLocation">spelling.txt</str>  -->
          </lst>
        </searchComponent>
      

      and issue this query request:

      curl "http://localhost:8983/solr/spellBug4?q=catle&indent=true"
      

      Which results in this NPE stack trace:

      <lst name="error">
        <str name="trace">java.lang.NullPointerException
              at org.apache.lucene.search.spell.SpellChecker.indexDictionary(SpellChecker.java:515)
              at org.apache.solr.spelling.FileBasedSpellChecker.build(FileBasedSpellChecker.java:68)
              at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:108)
              at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:185)
              at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
              at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
              at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
              at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
              at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
              at org.eclipse.jetty.server.Server.handle(Server.java:351)
              at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
              at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
              at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
              at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
              at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
              at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
              at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
              at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
              at java.lang.Thread.run(Unknown Source)
      </str> 
        <int name="code">500</int>
      </lst>
      

      There is a similar issue for IndexBasedSpellChecker. Change "FileBasedSpellChecker" to "IndexBasedSpellChecker" in the above solrcomfig.xml snippet and this similar but distinct NPE stack trace is produced:

      <lst name="error">
        <str name="trace">java.lang.NullPointerException
              at java.util.TreeMap.getEntry(Unknown Source)
              at java.util.TreeMap.get(Unknown Source)
              at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.terms(PerFieldPostingsFormat.java:215)
              at org.apache.lucene.index.MultiFields.getTerms(MultiFields.java:130)
              at org.apache.lucene.search.spell.HighFrequencyDictionary$HighFrequencyIterator.&lt;init&gt;(HighFrequencyDictionary.java:69)
              at org.apache.lucene.search.spell.HighFrequencyDictionary.getWordsIterator(HighFrequencyDictionary.java:59)
              at org.apache.lucene.search.spell.SpellChecker.indexDictionary(SpellChecker.java:515)
              at org.apache.solr.spelling.IndexBasedSpellChecker.build(IndexBasedSpellChecker.java:96)
              at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:108)
              at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:185)
              at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
              at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
              at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
              at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
              at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
              at org.eclipse.jetty.server.Server.handle(Server.java:351)
              at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
              at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
              at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
              at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
              at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
              at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
              at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
              at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
              at java.lang.Thread.run(Unknown Source)
      </str>
        <int name="code">500</int>
      </lst>
      

      In the case of IndexBasedSpellChecker, maybe the default for sourceLocation should be the "spelling" directory, since it takes a Lucene index directory rather than a text file like FileBasedSpellChecker. Or, give an error status that clearly states that the sourceLocation "directory" is required and missing.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jkrupan Jack Krupansky
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: