Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.3, 5.4
    • Fix Version/s: 5.5
    • Component/s: None
    • Labels:
      None

      Description

      I prepared a failing test. This worked for 4.x versions. This fails with different stacktrace on 5.1 and 5.0 versions. I'm not sure if it is related to Solr or Lucene. Since the stack trace is different before 5.2 maybe something changed here SOLR-5855?

      test code:

      public class Test extends SolrTestCaseJ4 {
        @BeforeClass
        public static void beforeClass() throws Exception {
          initCore("solrconfig.xml", "schema.xml");
        }
      
        @Test
        public void testConstantScoreQueryWithFilterPartOnly() {
          final String[] doc1 = {"id", "1", "location", "56.9485,24.0980"};
          assertU(adoc(doc1));
          assertU(commit());
      
          ModifiableSolrParams params = new ModifiableSolrParams();
          params.add("q", "{!geofilt sfield=\"location\" pt=\"56.9484,24.0981\" d=100}");
          params.add("hl", "true");
          params.add("hl.fl", "location");
          assertQ(req(params), "*[count(//doc)=1]", "count(//lst[@name='highlighting']/*)=1");
        }
      }
      

      solrconfig:

      <?xml version="1.0" ?>
      <config><luceneMatchVersion>${tests.luceneMatchVersion:LUCENE_CURRENT}</luceneMatchVersion>
          <dataDir>${solr.data.dir:}</dataDir>
          <directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.RAMDirectoryFactory}"/>
      
          <queryResponseWriter name="xml" default="true" class="solr.XMLResponseWriter" />
          <requestHandler name="/select" class="solr.SearchHandler" />
          <requestHandler name="/update" class="solr.UpdateRequestHandler"  />
      </config>
      

      schema:

      <schema name="example" version="1.1">
        <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
        <fieldType name="double" class="solr.TrieDoubleField" sortMissingLast="true" omitNorms="true"/>
        <fieldType name="int" class="solr.TrieIntField" sortMissingLast="true" omitNorms="true"/>
        <fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType" units="degrees" geo="true" />
      
        <field name="_version_" type="string"/>
        <field name="id" type="string" required="true"/>
        <field name="string_field" type="string"/>
        <field name="double_field" type="double"/>
        <field name="int_field" type="int"/>
        <field name="location" type="location_rpt" />
      
        <uniqueKey>id</uniqueKey>
      
        <defaultSearchField>string_field</defaultSearchField>
      </schema>
      

      This ends up with:

      Exception during query
      java.lang.RuntimeException
      	at org.apache.lucene.util.BytesRef.deepCopyOf(BytesRef.java:281)
      	at org.apache.lucene.analysis.tokenattributes.BytesTermAttributeImpl.copyTo(BytesTermAttributeImpl.java:51)
      	at org.apache.lucene.analysis.tokenattributes.BytesTermAttributeImpl.clone(BytesTermAttributeImpl.java:57)
      	at org.apache.lucene.util.AttributeSource$State.clone(AttributeSource.java:55)
      	at org.apache.lucene.util.AttributeSource.captureState(AttributeSource.java:280)
      	at org.apache.lucene.analysis.CachingTokenFilter.fillCache(CachingTokenFilter.java:96)
      	at org.apache.lucene.analysis.CachingTokenFilter.incrementToken(CachingTokenFilter.java:70)
      	at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:452)
      	at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:384)
      	at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:359)
      	at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:339)
      	at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getLeafContext(WeightedSpanTermExtractor.java:384)
      	at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:215)
      	at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:506)
      	at org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:219)
      	at org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:187)
      	at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:196)
      	at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:595)
      	at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:429)
      	at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
      	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:273)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
      	at org.apache.solr.util.TestHarness.query(TestHarness.java:311)
      	at org.apache.solr.util.TestHarness.query(TestHarness.java:293)
      	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:725)
      	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:718)
      

        Issue Links

          Activity

          Hide
          David Smiley added a comment -

          Thanks for reporting this. So the NPE is bad of course but I'm surprised or unclear on what you mean by this "worked" in 4x. What happens in 4x?

          Show
          David Smiley added a comment - Thanks for reporting this. So the NPE is bad of course but I'm surprised or unclear on what you mean by this "worked" in 4x. What happens in 4x?
          Hide
          Pawel Rog added a comment - - edited

          Thanks for your response. I mean that the code which you can see in integration test didn't throw NPE on Solr 4.x (eg. 4.9.0). Moreover the assertion passed.

          Let me know if you have any suggestions what can I execute to get more details which can be useful here. Thanks.

          Show
          Pawel Rog added a comment - - edited Thanks for your response. I mean that the code which you can see in integration test didn't throw NPE on Solr 4.x (eg. 4.9.0). Moreover the assertion passed. Let me know if you have any suggestions what can I execute to get more details which can be useful here. Thanks.
          Hide
          Pawel Rog added a comment - - edited

          I can reproduce this like below:

              final MemoryIndex indexer = new MemoryIndex();
              TokenStream tokenStream = h.getCore().getLatestSchema().getField("location").getType().getIndexAnalyzer().tokenStream("location", "56.9485 24.0980");
              tokenStream = new CachingTokenFilter(tokenStream);
              indexer.addField("test_field", tokenStream);
          

          When I don't use CachingTokenFilter everything works fine. So the code below doesn't throw NPE while the code above does:

              final MemoryIndex indexer = new MemoryIndex();
              TokenStream tokenStream = h.getCore().getLatestSchema().getField("location").getType().getIndexAnalyzer().tokenStream("location", "56.9485 24.0980");
              indexer.addField("test_field", tokenStream);
          
          Show
          Pawel Rog added a comment - - edited I can reproduce this like below: final MemoryIndex indexer = new MemoryIndex(); TokenStream tokenStream = h.getCore().getLatestSchema().getField( "location" ).getType().getIndexAnalyzer().tokenStream( "location" , "56.9485 24.0980" ); tokenStream = new CachingTokenFilter(tokenStream); indexer.addField( "test_field" , tokenStream); When I don't use CachingTokenFilter everything works fine. So the code below doesn't throw NPE while the code above does: final MemoryIndex indexer = new MemoryIndex(); TokenStream tokenStream = h.getCore().getLatestSchema().getField( "location" ).getType().getIndexAnalyzer().tokenStream( "location" , "56.9485 24.0980" ); indexer.addField( "test_field" , tokenStream);
          Hide
          David Smiley added a comment -

          Thanks Pawel. I filled LUCENE-6976 for this. I considered renaming/editing and moving this issue but the entire issue would need to be updated to reflect the actual problem so I just filed a new issue.

          I think this occurred as a side effect of LUCENE-6653/LUCENE-6652 in July.

          Show
          David Smiley added a comment - Thanks Pawel. I filled LUCENE-6976 for this. I considered renaming/editing and moving this issue but the entire issue would need to be updated to reflect the actual problem so I just filed a new issue. I think this occurred as a side effect of LUCENE-6653 / LUCENE-6652 in July.
          Hide
          Pawel Rog added a comment -

          Thanks David

          Show
          Pawel Rog added a comment - Thanks David
          Hide
          ASF subversion and git services added a comment -

          Commit 1724874 from David Smiley in branch 'dev/trunk'
          [ https://svn.apache.org/r1724874 ]

          LUCENE-6976 SOLR-8541: BytesTermAttributeImpl.copyTo could NPE.
          Could be triggered by trying to highlight a spatial RPT field.

          Show
          ASF subversion and git services added a comment - Commit 1724874 from David Smiley in branch 'dev/trunk' [ https://svn.apache.org/r1724874 ] LUCENE-6976 SOLR-8541 : BytesTermAttributeImpl.copyTo could NPE. Could be triggered by trying to highlight a spatial RPT field.
          Hide
          ASF subversion and git services added a comment -

          Commit 1724877 from David Smiley in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1724877 ]

          LUCENE-6976 SOLR-8541: BytesTermAttributeImpl.copyTo could NPE.
          Could be triggered by trying to highlight a spatial RPT field.

          Show
          ASF subversion and git services added a comment - Commit 1724877 from David Smiley in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1724877 ] LUCENE-6976 SOLR-8541 : BytesTermAttributeImpl.copyTo could NPE. Could be triggered by trying to highlight a spatial RPT field.
          Hide
          ASF subversion and git services added a comment -

          Commit 1725586 from Steve Rowe in branch 'dev/trunk'
          [ https://svn.apache.org/r1725586 ]

          SOLR-8541: remove unused imports

          Show
          ASF subversion and git services added a comment - Commit 1725586 from Steve Rowe in branch 'dev/trunk' [ https://svn.apache.org/r1725586 ] SOLR-8541 : remove unused imports
          Hide
          ASF subversion and git services added a comment -

          Commit 1725587 from Steve Rowe in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1725587 ]

          SOLR-8541: remove unused imports (merged trunk r1725586)

          Show
          ASF subversion and git services added a comment - Commit 1725587 from Steve Rowe in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1725587 ] SOLR-8541 : remove unused imports (merged trunk r1725586)
          Hide
          ASF subversion and git services added a comment -

          Commit 7d52c2523c7a4ff70612742b76b934a12b493331 in lucene-solr's branch refs/heads/branch_5_4 from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7d52c25 ]

          LUCENE-6976 SOLR-8541: BytesTermAttributeImpl.copyTo could NPE.
          Could be triggered by trying to highlight a spatial RPT field.

          git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x@1724877 13f79535-47bb-0310-9956-ffa450edef68

          Conflicts:
          lucene/CHANGES.txt
          solr/CHANGES.txt

          Show
          ASF subversion and git services added a comment - Commit 7d52c2523c7a4ff70612742b76b934a12b493331 in lucene-solr's branch refs/heads/branch_5_4 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7d52c25 ] LUCENE-6976 SOLR-8541 : BytesTermAttributeImpl.copyTo could NPE. Could be triggered by trying to highlight a spatial RPT field. git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x@1724877 13f79535-47bb-0310-9956-ffa450edef68 Conflicts: lucene/CHANGES.txt solr/CHANGES.txt

            People

            • Assignee:
              David Smiley
              Reporter:
              Pawel Rog
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development