Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7027

NumericTermAttribute throws IAE after NumericTokenStream is exhausted

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 5.5, 6.0
    • 5.5, 6.0
    • None
    • None
    • New

    Description

      This small test:

        public void testCloneFullPrecisionToken() throws Exception {
          FieldType fieldType = new FieldType(IntField.TYPE_NOT_STORED);
          fieldType.setNumericPrecisionStep(Integer.MAX_VALUE);
          Field field = new IntField("field", 17, fieldType);
          TokenStream tokenStream = new CachingTokenFilter(field.tokenStream(null, null));
          assertTrue(tokenStream.incrementToken());
        }
      

      hits this unexpected exception:

      There was 1 failure:
      1) testCloneFullPrecisionToken(org.apache.lucene.analysis.TestNumericTokenStream)
      java.lang.IllegalArgumentException: Illegal shift value, must be 0..31; got shift=2147483647
      	at __randomizedtesting.SeedInfo.seed([2E1E93EF810CB5F7:EF1304A849574BC7]:0)
      	at org.apache.lucene.util.NumericUtils.intToPrefixCodedBytes(NumericUtils.java:175)
      	at org.apache.lucene.util.NumericUtils.intToPrefixCoded(NumericUtils.java:133)
      	at org.apache.lucene.analysis.NumericTokenStream$NumericTermAttributeImpl.getBytesRef(NumericTokenStream.java:165)
      	at org.apache.lucene.analysis.NumericTokenStream$NumericTermAttributeImpl.clone(NumericTokenStream.java:217)
      	at org.apache.lucene.analysis.NumericTokenStream$NumericTermAttributeImpl.clone(NumericTokenStream.java:148)
      	at org.apache.lucene.util.AttributeSource$State.clone(AttributeSource.java:55)
      	at org.apache.lucene.util.AttributeSource.captureState(AttributeSource.java:288)
      	at org.apache.lucene.analysis.CachingTokenFilter.fillCache(CachingTokenFilter.java:96)
      	at org.apache.lucene.analysis.CachingTokenFilter.incrementToken(CachingTokenFilter.java:70)
      	at org.apache.lucene.analysis.TestNumericTokenStream.testCloneFullPrecisionToken(TestNumericTokenStream.java:138)
      

      because CachingTokenFilter expects that it can captureState after calling end but NumericTokenStream gets angry about this.

      Attachments

        1. LUCENE-7027-master.patch
          5 kB
          Uwe Schindler
        2. LUCENE-7027-master.patch
          4 kB
          Uwe Schindler
        3. LUCENE-7027-master.patch
          2 kB
          Uwe Schindler

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            uschindler Uwe Schindler
            mikemccand Michael McCandless
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment