Lucene - Core
  1. Lucene - Core
  2. LUCENE-3065

NumericField should be stored in binary format in index (matching Solr's format)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2, 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      (Spinoff of LUCENE-3001)

      Today when writing stored fields we don't record that the field was a NumericField, and so at IndexReader time you get back an "ordinary" Field and your number has turned into a string. See https://issues.apache.org/jira/browse/LUCENE-1701?focusedCommentId=12721972&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12721972

      We have spare bits already in stored fields, so, we should use one to record that the field is numeric, and then encode the numeric field in Solr's more-compact binary format.

      A nice side-effect is we fix the long standing issue that you don't get a NumericField back when loading your document.

      1. LUCENE-3065.patch
        32 kB
        Uwe Schindler
      2. LUCENE-3065.patch
        31 kB
        Uwe Schindler
      3. LUCENE-3065.patch
        26 kB
        Uwe Schindler
      4. LUCENE-3065.patch
        22 kB
        Uwe Schindler
      5. LUCENE-3065.patch
        22 kB
        Uwe Schindler
      6. LUCENE-3065.patch
        32 kB
        Uwe Schindler
      7. LUCENE-3065.patch
        32 kB
        Uwe Schindler
      8. LUCENE-3065.patch
        29 kB
        Uwe Schindler
      9. LUCENE-3065.patch
        28 kB
        Michael McCandless
      10. LUCENE-3065-trunk.patch
        33 kB
        Uwe Schindler

        Issue Links

          Activity

          Hide
          Robert Muir added a comment -

          Bulk closing for 3.2

          Show
          Robert Muir added a comment - Bulk closing for 3.2
          Hide
          Uwe Schindler added a comment -

          Committed trunk revision: 1100526

          Show
          Uwe Schindler added a comment - Committed trunk revision: 1100526
          Hide
          Uwe Schindler added a comment -

          This is the patch for trunk. Will commit soon!

          Show
          Uwe Schindler added a comment - This is the patch for trunk. Will commit soon!
          Hide
          Uwe Schindler added a comment -

          Committed 3.x revision: 1100480

          Now forward-porting to trunk...

          Show
          Uwe Schindler added a comment - Committed 3.x revision: 1100480 Now forward-porting to trunk...
          Hide
          Robert Muir added a comment -

          ahh sorry I missed that. patch looks good to me though!

          Show
          Robert Muir added a comment - ahh sorry I missed that. patch looks good to me though!
          Hide
          Uwe Schindler added a comment -
          Show
          Uwe Schindler added a comment - Just to note: We also need to change the Forrest index format documentation! I already commented on that https://issues.apache.org/jira/browse/LUCENE-3065?focusedCommentId=13028718&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13028718
          Hide
          Robert Muir added a comment -

          I just add this TODO here:
          Don't forget to add a new 3.1 index format to TestBackwardsCompatibility!

          Can we also update the description of the bits in fileformats.html?

          Show
          Robert Muir added a comment - I just add this TODO here: Don't forget to add a new 3.1 index format to TestBackwardsCompatibility! Can we also update the description of the bits in fileformats.html?
          Hide
          Michael McCandless added a comment -

          Looks great Uwe! Awesome to finally get NumericField back at search time...

          Show
          Michael McCandless added a comment - Looks great Uwe! Awesome to finally get NumericField back at search time...
          Hide
          Uwe Schindler added a comment -

          New patch with the changes proposed before (no more instanceof chains).

          I think this is now ready to commit.

          Show
          Uwe Schindler added a comment - New patch with the changes proposed before (no more instanceof chains). I think this is now ready to commit.
          Hide
          Uwe Schindler added a comment -

          I just add this TODO here:
          Don't forget to add a new 3.1 index format to TestBackwardsCompatibility!

          Show
          Uwe Schindler added a comment - I just add this TODO here: Don't forget to add a new 3.1 index format to TestBackwardsCompatibility!
          Hide
          Uwe Schindler added a comment -

          Updated patch with some improvements:

          • NumericField now lazy inits the NumericTokenStream only when tokenStreamValue() is caled for the first time. This speeds up stored fields reading, as the TokenStream is generally not needed in that case.
          • I currently dont like the instanceof chains in FieldsWriter and this lazy init code. Maybe NumericField and NumericTokenStream should define an enum type for the value so you can call NumericField.getValueType() - does anybody have a better idea?
          • Improved JavaDocs for NumericField to reflect the new stored fields format
          Show
          Uwe Schindler added a comment - Updated patch with some improvements: NumericField now lazy inits the NumericTokenStream only when tokenStreamValue() is caled for the first time. This speeds up stored fields reading, as the TokenStream is generally not needed in that case. I currently dont like the instanceof chains in FieldsWriter and this lazy init code. Maybe NumericField and NumericTokenStream should define an enum type for the value so you can call NumericField.getValueType() - does anybody have a better idea? Improved JavaDocs for NumericField to reflect the new stored fields format
          Hide
          Uwe Schindler added a comment -

          Earwin: The long-term plan for flexible indexing is to make also stored fields flexible. For now its not possible, so NumericFields are handled separately. In the future, this might be a stored fields codec.

          Show
          Uwe Schindler added a comment - Earwin: The long-term plan for flexible indexing is to make also stored fields flexible. For now its not possible, so NumericFields are handled separately. In the future, this might be a stored fields codec.
          Hide
          Earwin Burrfoot added a comment -

          It's sad NumericFields are hardbaked into index format.

          Eg - I have some fields that are similar to Numeric in that they are 'stringified' binary structures, and they can't become first-class in the same manner as Numeric.

          Show
          Earwin Burrfoot added a comment - It's sad NumericFields are hardbaked into index format. Eg - I have some fields that are similar to Numeric in that they are 'stringified' binary structures, and they can't become first-class in the same manner as Numeric.
          Hide
          Uwe Schindler added a comment - - edited

          Revert of deletion of Mike's first comment (sorry)

          Patch against 3.x.

          I moved the to/from byte[] methods from Solr's TrieField into Lucene's NumericUtils, and fixed FieldsWriter/Reader to use free bits in the field's flags to know if the field is Numeric, and which type.

          I added a random test case to verify we now get the right NumericField back, when we stored NumericField during indexing.

          Old indices are handled fine (you'll get a String-ified Field back like you did before).

          Spookily, nothing failed in Solr... I assume there's somewhere in Solr that must now be fixed to handle the fact that a field can come back as NumericField? Anyone know where...?)

          Show
          Uwe Schindler added a comment - - edited Revert of deletion of Mike's first comment (sorry) Patch against 3.x. I moved the to/from byte[] methods from Solr's TrieField into Lucene's NumericUtils, and fixed FieldsWriter/Reader to use free bits in the field's flags to know if the field is Numeric, and which type. I added a random test case to verify we now get the right NumericField back, when we stored NumericField during indexing. Old indices are handled fine (you'll get a String-ified Field back like you did before). Spookily, nothing failed in Solr... I assume there's somewhere in Solr that must now be fixed to handle the fact that a field can come back as NumericField? Anyone know where...?)
          Hide
          Uwe Schindler added a comment -

          Sorry my browser or JIRA deleted wrong comments, so I removed one from me and one from Mike - Sorry.

          Show
          Uwe Schindler added a comment - Sorry my browser or JIRA deleted wrong comments, so I removed one from me and one from Mike - Sorry.
          Hide
          Uwe Schindler added a comment -

          I started a new issue in Solr for the changes there: SOLR-2497

          Show
          Uwe Schindler added a comment - I started a new issue in Solr for the changes there: SOLR-2497
          Hide
          Uwe Schindler added a comment - - edited

          I added some javadocs to Document class:

          • getField() / getFields() is deprecated [we may change this in LUCENE-2310]

          Some thoughts:

          • maybe we should make getField()/getFields() simply return null or does not include the Field into the returned array, if its not instanceof Field? We can add that to documentation, that lazy loaded and numerical fields are not returned.
          • I would also like to add a method Document.getNumericValue(s), that returns Number[] or Number like the NumericField one. Like above getField() it can return null/empty array if the field name has no numeric Fields?

          The CHANGES entry may also be extended, currently it under "bugs" - we shold move.

          Show
          Uwe Schindler added a comment - - edited I added some javadocs to Document class: getField() / getFields() is deprecated [we may change this in LUCENE-2310] Some thoughts: maybe we should make getField()/getFields() simply return null or does not include the Field into the returned array, if its not instanceof Field? We can add that to documentation, that lazy loaded and numerical fields are not returned. I would also like to add a method Document.getNumericValue(s), that returns Number[] or Number like the NumericField one. Like above getField() it can return null/empty array if the field name has no numeric Fields? The CHANGES entry may also be extended, currently it under "bugs" - we shold move.
          Hide
          Michael McCandless added a comment -

          Patch looks great Uwe!

          I think we should deprecate Document.getField? And advertise in CHANGES that this is an [intentional] BW break, ie, you can no longer .getField if it's a NumericField (you'll hit CCE, just like you already do for lazy fields)? I think that's the lesser evil here?

          Show
          Michael McCandless added a comment - Patch looks great Uwe! I think we should deprecate Document.getField? And advertise in CHANGES that this is an [intentional] BW break, ie, you can no longer .getField if it's a NumericField (you'll hit CCE, just like you already do for lazy fields)? I think that's the lesser evil here?
          Hide
          Uwe Schindler added a comment -

          Moved test to TestFieldsReader

          Show
          Uwe Schindler added a comment - Moved test to TestFieldsReader
          Hide
          Uwe Schindler added a comment -

          Just to note: We also need to change the Forrest index format documentation!

          Show
          Uwe Schindler added a comment - Just to note: We also need to change the Forrest index format documentation!
          Hide
          Uwe Schindler added a comment -

          Next iteration:

          Reverted changes in Solr (they should come later), Lucene instead uses natively IndexInput and IndexOutput to write/read ints and longs.

          Solr's changes are completely unrelated.

          Show
          Uwe Schindler added a comment - Next iteration: Reverted changes in Solr (they should come later), Lucene instead uses natively IndexInput and IndexOutput to write/read ints and longs. Solr's changes are completely unrelated.
          Hide
          Uwe Schindler added a comment - - edited

          This patch adds some refactoring because FieldSelectorResult is an enum since 3.0, so the (slow) queue of if-statements can be replaced by a fast switch.

          Also some minor comments and a missing & 0xFF when casting byte to int.

          Show
          Uwe Schindler added a comment - - edited This patch adds some refactoring because FieldSelectorResult is an enum since 3.0, so the (slow) queue of if-statements can be replaced by a fast switch. Also some minor comments and a missing & 0xFF when casting byte to int.
          Hide
          Uwe Schindler added a comment -

          New patch, previous one had a leftover unused constant from Mike's patch.

          Show
          Uwe Schindler added a comment - New patch, previous one had a leftover unused constant from Mike's patch.
          Hide
          Uwe Schindler added a comment -

          More refactoring:

          • Now NumericFields also reproduce the indexed/omitNorms/omitTF settings - only precStep cannot be reproduced
          • Cut over to int instead of byte, this removes lots of casting in FieldsReader
          Show
          Uwe Schindler added a comment - More refactoring: Now NumericFields also reproduce the indexed/omitNorms/omitTF settings - only precStep cannot be reproduced Cut over to int instead of byte, this removes lots of casting in FieldsReader
          Hide
          Michael McCandless added a comment -

          Patch looks great Uwe! Except we need to resolve this Field/Fieldable/AbstractField. Probably we should go and finish LUCENE-2310...

          Show
          Michael McCandless added a comment - Patch looks great Uwe! Except we need to resolve this Field/Fieldable/AbstractField. Probably we should go and finish LUCENE-2310 ...
          Hide
          Uwe Schindler added a comment -

          Here the patch with my changes

          Show
          Uwe Schindler added a comment - Here the patch with my changes
          Hide
          Uwe Schindler added a comment -

          This gives us more freedom in future, as we are limit to completely 8 bits, 3 are already used - this only adds 3 more not 4.

          By the way, for performance reasons all constants should be declared as int not byte, as the byte read from index is already an int.

          Show
          Uwe Schindler added a comment - This gives us more freedom in future, as we are limit to completely 8 bits, 3 are already used - this only adds 3 more not 4. By the way, for performance reasons all constants should be declared as int not byte, as the byte read from index is already an int.
          Hide
          Uwe Schindler added a comment -

          Mike:
          I reviewed the patch again: You are currently using 3 bits already. 1 bit is solely for detecting numerics, the other two are the type.

          In my opinion, to check if its a numeric field, use a MASK of 3 bits and check for !=0. As soon as any bit in this mask is set, its numeric. The actual numeric fields have values !=0:

          private static final int _NUMERIC_BIT_SHIFT = 3;
          static final byte FIELD_IS_NUMERIC_MASK = 0x07 << _NUMERIC_BIT_SHIFT;
          
          static final byte FIELD_IS_NUMERIC_INT = 1 << _NUMERIC_BIT_SHIFT;
          static final byte FIELD_IS_NUMERIC_LONG = 2 << _NUMERIC_BIT_SHIFT;
          static final byte FIELD_IS_NUMERIC_FLOAT = 3 << _NUMERIC_BIT_SHIFT;
          static final byte FIELD_IS_NUMERIC_DOUBLE = 4 << _NUMERIC_BIT_SHIFT;
          // unused: static final byte FIELD_IS_NUMERIC_SHORT = 5 << _NUMERIC_BIT_SHIFT;
          // unused: static final byte FIELD_IS_NUMERIC_BYTE = 6 << _NUMERIC_BIT_SHIFT;
          // and we have still one more over :-)  7 << _NUMERIC_BIT_SHIFT
          
          // check if field is numeric:
          if ((bits & FIELD_IS_NUMERIC_MASK) != 0) {}
          
          // parse type:
          switch (bits & FIELD_IS_NUMERIC_MASK) {
            case FIELD_IS_NUMERIC_INT: ...
          }
          
          Show
          Uwe Schindler added a comment - Mike: I reviewed the patch again: You are currently using 3 bits already. 1 bit is solely for detecting numerics, the other two are the type. In my opinion, to check if its a numeric field, use a MASK of 3 bits and check for !=0. As soon as any bit in this mask is set, its numeric. The actual numeric fields have values !=0: private static final int _NUMERIC_BIT_SHIFT = 3; static final byte FIELD_IS_NUMERIC_MASK = 0x07 << _NUMERIC_BIT_SHIFT; static final byte FIELD_IS_NUMERIC_INT = 1 << _NUMERIC_BIT_SHIFT; static final byte FIELD_IS_NUMERIC_LONG = 2 << _NUMERIC_BIT_SHIFT; static final byte FIELD_IS_NUMERIC_FLOAT = 3 << _NUMERIC_BIT_SHIFT; static final byte FIELD_IS_NUMERIC_DOUBLE = 4 << _NUMERIC_BIT_SHIFT; // unused: static final byte FIELD_IS_NUMERIC_SHORT = 5 << _NUMERIC_BIT_SHIFT; // unused: static final byte FIELD_IS_NUMERIC_BYTE = 6 << _NUMERIC_BIT_SHIFT; // and we have still one more over :-) 7 << _NUMERIC_BIT_SHIFT // check if field is numeric: if ((bits & FIELD_IS_NUMERIC_MASK) != 0) {} // parse type: switch (bits & FIELD_IS_NUMERIC_MASK) { case FIELD_IS_NUMERIC_INT: ... }
          Hide
          Chris Male added a comment -

          Yeah there is an element of truth to that except I'm not convinced we need to have such a complicated hierarchy (although I've since been thinking about field definitions coming from different sources, so maybe an interface is best). But yes, Field is a mess and I've been trying to clean that out too.

          Show
          Chris Male added a comment - Yeah there is an element of truth to that except I'm not convinced we need to have such a complicated hierarchy (although I've since been thinking about field definitions coming from different sources, so maybe an interface is best). But yes, Field is a mess and I've been trying to clean that out too.
          Hide
          Yonik Seeley added a comment -

          I then deprecated all the Fieldable methods in Document.

          Hmmm, I thought Fieldable was a step forward. The Field class is the worst of the bunch!

          Show
          Yonik Seeley added a comment - I then deprecated all the Fieldable methods in Document. Hmmm, I thought Fieldable was a step forward. The Field class is the worst of the bunch!
          Hide
          Chris Male added a comment -

          The Field/Fieldable/AbstractField problem is what I've been addressing in LUCENE-2310. There I took the step of making NumericField extend Field, with a series of unsupported fields. This seemed easiest to do particularly with FieldType in mind. I then deprecated all the Fieldable methods in Document.

          Show
          Chris Male added a comment - The Field/Fieldable/AbstractField problem is what I've been addressing in LUCENE-2310 . There I took the step of making NumericField extend Field, with a series of unsupported fields. This seemed easiest to do particularly with FieldType in mind. I then deprecated all the Fieldable methods in Document.
          Hide
          Michael McCandless added a comment -

          Ugh! Field/Fieldable/AbstractField strikes again.... hmm not sure what to do.

          Show
          Michael McCandless added a comment - Ugh! Field/Fieldable/AbstractField strikes again.... hmm not sure what to do.
          Hide
          Uwe Schindler added a comment - - edited

          There is still a problem - first the good news:

          • If user calls Document.get(field), the returned string is as before, so there is no break at all. The reason is the implementation of NumericField.stringValue(), it returns what the user is used to from 3.0
          • If a user calls getFieldable(field) all is fine, too. The only change is that it could return NumericField now. If the user simply calls stringValue() all is identical to 3.0

          Problems start with:

          • If user calls Document.getField(name) it returns Field (internally it casts the getFieldable()) result to Field. But NumericField does not subclass Field -> ClassCastException.

          How to handle this?

          • Maybe change those methods to return AbstractField, but thats a binary break and users will complain, because not everything works as expected
          • Make NumericField subclass Field (and Field is unfinalized) - thats a bad idea, because Field has too many methods / members that are out of scope
          • Deprecate Document.getField() and make it internally do an instanceof check, if it gets NumericField transform to a backwards-compatible Field? - This method is already broken. If you request Lazy field loading it also throws ClassCastEx (e.g. LUCENE-609).

          Not sure how to proceed. Else the patch looks fine. I think simply ignoring LazyField loading is fine, as numeric fields are a maximum of 8 bytes.... Else we would need LazyNumericField

          Show
          Uwe Schindler added a comment - - edited There is still a problem - first the good news: If user calls Document.get(field), the returned string is as before, so there is no break at all. The reason is the implementation of NumericField.stringValue(), it returns what the user is used to from 3.0 If a user calls getFieldable(field) all is fine, too. The only change is that it could return NumericField now. If the user simply calls stringValue() all is identical to 3.0 Problems start with: If user calls Document.getField(name) it returns Field (internally it casts the getFieldable()) result to Field. But NumericField does not subclass Field -> ClassCastException. How to handle this? Maybe change those methods to return AbstractField, but thats a binary break and users will complain, because not everything works as expected Make NumericField subclass Field (and Field is unfinalized) - thats a bad idea, because Field has too many methods / members that are out of scope Deprecate Document.getField() and make it internally do an instanceof check, if it gets NumericField transform to a backwards-compatible Field? - This method is already broken. If you request Lazy field loading it also throws ClassCastEx (e.g. LUCENE-609 ). Not sure how to proceed. Else the patch looks fine. I think simply ignoring LazyField loading is fine, as numeric fields are a maximum of 8 bytes.... Else we would need LazyNumericField
          Hide
          Uwe Schindler added a comment -

          Mike: One thing about the bitmask and the 4 values. There is also an issue open to extend NumericField by byte and short. Maybe we should reserve 3 bits instead of 2 for the numeric field type - so 0x70 instead of 0x30 as mask? I just want to reseve this one extra bit, so we dont need to do any dumb masks and values later, if we extend.

          About the index format change:
          As described above, for Solr it's not a problem. New fields are always indexed using NumericField. On the query side, when Document.getField is called, it could simply check the return value with instanceof. If the getter returns not a NumericField, Solr knows that it's binary and can decode manually. This would safe backwards.

          Else its no break at all if we support both stored field formats during indexing somehow (in Lucene its string, returning a String Field or new binary NumericField). The index format itsself does not change generally (no need to bump version numbers, as we only use unused bits?)

          Show
          Uwe Schindler added a comment - Mike: One thing about the bitmask and the 4 values. There is also an issue open to extend NumericField by byte and short. Maybe we should reserve 3 bits instead of 2 for the numeric field type - so 0x70 instead of 0x30 as mask? I just want to reseve this one extra bit, so we dont need to do any dumb masks and values later, if we extend. About the index format change: As described above, for Solr it's not a problem. New fields are always indexed using NumericField. On the query side, when Document.getField is called, it could simply check the return value with instanceof. If the getter returns not a NumericField, Solr knows that it's binary and can decode manually. This would safe backwards. Else its no break at all if we support both stored field formats during indexing somehow (in Lucene its string, returning a String Field or new binary NumericField). The index format itsself does not change generally (no need to bump version numbers, as we only use unused bits?)
          Hide
          Yonik Seeley added a comment -

          I'll take a crack at it... but I don't think its necessary in the first pass

          Should we try to accept both (binary or numeric field coming back) so this isn't a needless index format break, or is there another lucene index format break in the cards soon anyway?

          Show
          Yonik Seeley added a comment - I'll take a crack at it... but I don't think its necessary in the first pass Should we try to accept both (binary or numeric field coming back) so this isn't a needless index format break, or is there another lucene index format break in the cards soon anyway?
          Hide
          Ryan McKinley added a comment -

          If so, can you take a crack at it? Thanks. Or, we can postpone... not necessary for this initial cutover.

          I'll take a crack at it... but I don't think its necessary in the first pass

          Show
          Ryan McKinley added a comment - If so, can you take a crack at it? Thanks. Or, we can postpone... not necessary for this initial cutover. I'll take a crack at it... but I don't think its necessary in the first pass
          Hide
          Michael McCandless added a comment -

          Uwe: I agree, I'll use BytesRef in trunk.

          Ryan: OK. Should we try to fix that w/ this issue? If so, can you take a crack at it? Thanks. Or, we can postpone... not necessary for this initial cutover.

          Show
          Michael McCandless added a comment - Uwe: I agree, I'll use BytesRef in trunk. Ryan: OK. Should we try to fix that w/ this issue? If so, can you take a crack at it? Thanks. Or, we can postpone... not necessary for this initial cutover.
          Hide
          Ryan McKinley added a comment -

          Is there some reason...?

          Solr did its own encoding/decoding so that it could store a binary field – with this patch, that is not necessary anymore.

          Show
          Ryan McKinley added a comment - Is there some reason...? Solr did its own encoding/decoding so that it could store a binary field – with this patch, that is not necessary anymore.
          Hide
          Michael McCandless added a comment -

          Thats easy to understand: Solr does not use NumericField at all. It produces a NumericTokenStream and indexes it like any other analyzer. The byte[] field is indexed as a separate Field with only store=true and binary.

          This is what I wanted to say with my last comment.

          Ahhhh, OK. So, not spooky.

          We should eventually fix that; shouldn't Solr just use NumericField instead of doing this encode/decode itself? Is there some reason...?

          Show
          Michael McCandless added a comment - Thats easy to understand: Solr does not use NumericField at all. It produces a NumericTokenStream and indexes it like any other analyzer. The byte[] field is indexed as a separate Field with only store=true and binary. This is what I wanted to say with my last comment. Ahhhh, OK. So, not spooky. We should eventually fix that; shouldn't Solr just use NumericField instead of doing this encode/decode itself? Is there some reason...?
          Hide
          Uwe Schindler added a comment -

          Aboutthe patch: Maybe the byte[] returning methods in NumericUtils should use BytesRef and reuse that for storing (applies to trunk)?

          Show
          Uwe Schindler added a comment - Aboutthe patch: Maybe the byte[] returning methods in NumericUtils should use BytesRef and reuse that for storing (applies to trunk)?
          Hide
          Uwe Schindler added a comment -

          Spookily, nothing failed in Solr... I assume there's somewhere in Solr
          that must now be fixed to handle the fact that a field can come back
          as NumericField? Anyone know where...?

          Thats easy to understand: Solr does not use NumericField at all. It produces a NumericTokenStream and indexes it like any other analyzer. The byte[] field is indexed as a separate Field with only store=true and binary.

          This is what I wanted to say with my last comment.

          Show
          Uwe Schindler added a comment - Spookily, nothing failed in Solr... I assume there's somewhere in Solr that must now be fixed to handle the fact that a field can come back as NumericField? Anyone know where...? Thats easy to understand: Solr does not use NumericField at all. It produces a NumericTokenStream and indexes it like any other analyzer. The byte[] field is indexed as a separate Field with only store=true and binary. This is what I wanted to say with my last comment.

            People

            • Assignee:
              Uwe Schindler
              Reporter:
              Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development