Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2074

Use a separate JFlex generated Unicode 4 by Java 5 compatible StandardTokenizer

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      The current trunk version of StandardTokenizerImpl was generated by Java 1.4 (according to the warning). In Java 3.0 we switch to Java 1.5, so we should regenerate the file.

      After regeneration the Tokenizer behaves different for some characters. Because of that we should only use the new TokenizerImpl when Version.LUCENE_30 or LUCENE_31 is used as matchVersion.

        Attachments

        1. jflex-1.4.1-vs-1.5-snapshot.diff
          5 kB
          Uwe Schindler
        2. jflexwarning.patch
          0.7 kB
          Mark Miller
        3. LUCENE-2074.patch
          69 kB
          Uwe Schindler
        4. LUCENE-2074.patch
          67 kB
          Uwe Schindler
        5. LUCENE-2074.patch
          65 kB
          Uwe Schindler
        6. LUCENE-2074.patch
          63 kB
          Uwe Schindler
        7. LUCENE-2074.patch
          65 kB
          Uwe Schindler
        8. LUCENE-2074.patch
          55 kB
          Uwe Schindler
        9. LUCENE-2074.patch
          40 kB
          Uwe Schindler
        10. LUCENE-2074.patch
          41 kB
          Uwe Schindler
        11. LUCENE-2074.patch
          40 kB
          Uwe Schindler
        12. LUCENE-2074.patch
          40 kB
          Uwe Schindler
        13. LUCENE-2074-lucene30.patch
          3 kB
          Uwe Schindler

          Issue Links

            Activity

              People

              • Assignee:
                thetaphi Uwe Schindler
                Reporter:
                thetaphi Uwe Schindler
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: