Lucene - Core
  1. Lucene - Core
  2. LUCENE-2295

Create a MaxFieldLengthAnalyzer to wrap any other Analyzer and provide the same functionality as MaxFieldLength provided on IndexWriter

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: modules/analysis
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      A spinoff from LUCENE-2294. Instead of asking the user to specify on IndexWriter his requested MFL limit, we can get rid of this setting entirely by providing an Analyzer which will wrap any other Analyzer and its TokenStream with a TokenFilter that keeps track of the number of tokens produced and stop when the limit has reached.

      This will remove any count tracking in IW's indexing, which is done even if I specified UNLIMITED for MFL.

      Let's try to do it for 3.1.

      1. LUCENE-2295-trunk.patch
        13 kB
        Uwe Schindler
      2. LUCENE-2295-2-trunk.patch
        24 kB
        Shai Erera
      3. LUCENE-2295-2-3x.patch
        23 kB
        Shai Erera
      4. LUCENE-2295.patch
        11 kB
        Uwe Schindler

        Activity

        Shai Erera created issue -
        Uwe Schindler made changes -
        Field Original Value New Value
        Assignee Uwe Schindler [ thetaphi ]
        Uwe Schindler made changes -
        Attachment LUCENE-2295.patch [ 12438099 ]
        Robert Muir made changes -
        Component/s contrib/analyzers [ 12312333 ]
        Uwe Schindler made changes -
        Attachment LUCENE-2295-trunk.patch [ 12445651 ]
        Uwe Schindler made changes -
        Fix Version/s 3.1 [ 12314822 ]
        Uwe Schindler made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Lucene Fields [New] [New, Patch Available]
        Resolution Fixed [ 1 ]
        Robert Muir made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Shai Erera made changes -
        Attachment LUCENE-2295-2-3x.patch [ 12468631 ]
        Shai Erera made changes -
        Attachment LUCENE-2295-2-trunk.patch [ 12468639 ]
        Shai Erera made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Mark Thomas made changes -
        Workflow jira [ 12500934 ] Default workflow, editable Closed status [ 12564246 ]
        Mark Thomas made changes -
        Workflow Default workflow, editable Closed status [ 12564246 ] jira [ 12584037 ]
        Grant Ingersoll made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Shai Erera made changes -
        Component/s modules/analysis [ 12310230 ]
        Component/s contrib/analyzers [ 12312333 ]

          People

          • Assignee:
            Uwe Schindler
            Reporter:
            Shai Erera
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development