Lucene - Core
  1. Lucene - Core
  2. LUCENE-3307

don't require an analyzer, if all fields are NOT_ANALYZED

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      This seems wierd, if you analyze only NOT_ANALYZED fields, you must have an analyzer (null will not work)
      because documentsinverter wants it for things like offsetGap

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        4d 10h 44m 1 Shai Erera 17/Jul/11 06:39
        Resolved Resolved Closed Closed
        133d 6h 52m 1 Uwe Schindler 27/Nov/11 12:31
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Shai Erera made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Lucene Fields [New] [New, Patch Available]
        Fix Version/s 3.4 [ 12316675 ]
        Fix Version/s 4.0 [ 12314025 ]
        Resolution Fixed [ 1 ]
        Hide
        Shai Erera added a comment -

        Committed revision 1147530 (trunk).
        Committed revision 1147532 (3x).

        I did not include a CHANGES entry, because we don't want to advertise null as a supported value for IW.

        Thanks Robert !

        Show
        Shai Erera added a comment - Committed revision 1147530 (trunk). Committed revision 1147532 (3x). I did not include a CHANGES entry, because we don't want to advertise null as a supported value for IW. Thanks Robert !
        Shai Erera made changes -
        Assignee Shai Erera [ shaie ]
        Hide
        Shai Erera added a comment -

        Yes, I'll commit this. Thanks !

        Show
        Shai Erera added a comment - Yes, I'll commit this. Thanks !
        Hide
        Robert Muir added a comment -

        Shai you want to take this? I am working on something else at the moment!

        Show
        Robert Muir added a comment - Shai you want to take this? I am working on something else at the moment!
        Hide
        Shai Erera added a comment -

        Ok I agree. +1 to commit.

        Show
        Shai Erera added a comment - Ok I agree. +1 to commit.
        Hide
        Robert Muir added a comment -

        At that point this "problem" of having to pass some random Analyzer or null should go away, because you should then only be required to pass an Analyzer to FieldTypes that do analyze their field text.

        Right, and the last thing i want to see is some sophisticated backwards layer / crappy hacks because we enabled null on this issue!

        Basically, I think we should make it work right now, but undocumented and unspecified. If you pass Analyzer=null to IW you need to be prepared for NPE.

        Show
        Robert Muir added a comment - At that point this "problem" of having to pass some random Analyzer or null should go away, because you should then only be required to pass an Analyzer to FieldTypes that do analyze their field text. Right, and the last thing i want to see is some sophisticated backwards layer / crappy hacks because we enabled null on this issue! Basically, I think we should make it work right now, but undocumented and unspecified. If you pass Analyzer=null to IW you need to be prepared for NPE.
        Hide
        Michael McCandless added a comment -

        Also, remember we are wanting to fully decouple indexing from analysis (LUCENE-2309) at which point IndexWriter(Config) will no longer even see an Analyzer; the use of an Analyzer (or any other app-specific pipeline/capability/etc.) will be hidden to IW under the IndexableField API. At that point this "problem" of having to pass some random Analyzer or null should go away, because you should then only be required to pass an Analyzer to FieldTypes that do analyze their field text.

        Show
        Michael McCandless added a comment - Also, remember we are wanting to fully decouple indexing from analysis ( LUCENE-2309 ) at which point IndexWriter(Config) will no longer even see an Analyzer; the use of an Analyzer (or any other app-specific pipeline/capability/etc.) will be hidden to IW under the IndexableField API. At that point this "problem" of having to pass some random Analyzer or null should go away, because you should then only be required to pass an Analyzer to FieldTypes that do analyze their field text.
        Hide
        Robert Muir added a comment -

        Robert, perhaps we should add an explicit test, in core, which ensures we can pass a null Analyzer to IWC when we index only NOT_ANALYZED and NumericFields?

        I don't like the idea of any kind of test or documentation that says we explicitly support null.
        Then, suddenly null is an officially supported value for Analyzer and people start wanting it in other places.

        If we must have a test here to proceed, I would instead prefer to cancel as Won't fix.

        Show
        Robert Muir added a comment - Robert, perhaps we should add an explicit test, in core, which ensures we can pass a null Analyzer to IWC when we index only NOT_ANALYZED and NumericFields? I don't like the idea of any kind of test or documentation that says we explicitly support null. Then, suddenly null is an officially supported value for Analyzer and people start wanting it in other places. If we must have a test here to proceed, I would instead prefer to cancel as Won't fix.
        Hide
        Shai Erera added a comment -

        +1 !

        This is also weird if you index only NumericFields. You need to pass whatever random analyzer that crosses your mind at the moment, just so that it's used to get the offset gap and posIncr.

        Robert, perhaps we should add an explicit test, in core, which ensures we can pass a null Analyzer to IWC when we index only NOT_ANALYZED and NumericFields?

        Show
        Shai Erera added a comment - +1 ! This is also weird if you index only NumericFields. You need to pass whatever random analyzer that crosses your mind at the moment, just so that it's used to get the offset gap and posIncr. Robert, perhaps we should add an explicit test, in core, which ensures we can pass a null Analyzer to IWC when we index only NOT_ANALYZED and NumericFields?
        Hide
        Uwe Schindler added a comment -

        +1

        Show
        Uwe Schindler added a comment - +1
        Robert Muir made changes -
        Field Original Value New Value
        Attachment LUCENE-3307.patch [ 12486213 ]
        Robert Muir created issue -

          People

          • Assignee:
            Shai Erera
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development