Lucene - Core
  1. Lucene - Core
  2. LUCENE-6884

Analyzer.tokenStream() shouldn't throw IOException

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      I'm guessing that in the past, calling Analyzer.tokenStream() would call TokenStream.reset() somewhere downstream, meaning that we had to deal with IOExceptions. However, tokenstreams are created entirely lazily now, so this is unnecessary.

      1. LUCENE-6884.patch
        20 kB
        Alan Woodward

        Activity

        Hide
        Alan Woodward added a comment -

        Patch, mostly just removing now-redundant try-catch blocks in tests. This also removes IOExceptions from Tokenizer.setReader()

        The only place that was actually throwing an IOException inside setReader was AbstractSpatialPrefixTreeFieldType, which I've changed to wrap as a RuntimeException. It's apparently only used for the Solr analysis UI, so I think this should be fine, but it would be good if David Smiley could double-check that.

        Show
        Alan Woodward added a comment - Patch, mostly just removing now-redundant try-catch blocks in tests. This also removes IOExceptions from Tokenizer.setReader() The only place that was actually throwing an IOException inside setReader was AbstractSpatialPrefixTreeFieldType, which I've changed to wrap as a RuntimeException. It's apparently only used for the Solr analysis UI, so I think this should be fine, but it would be good if David Smiley could double-check that.
        Hide
        David Smiley added a comment -

        It's fine.

        Show
        David Smiley added a comment - It's fine.
        Hide
        ASF subversion and git services added a comment -

        Commit 1712731 from Alan Woodward in branch 'dev/trunk'
        [ https://svn.apache.org/r1712731 ]

        LUCENE-6884: Analyzer.tokenStream() and Tokenizer.setReader() don't need to throw IOException

        Show
        ASF subversion and git services added a comment - Commit 1712731 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1712731 ] LUCENE-6884 : Analyzer.tokenStream() and Tokenizer.setReader() don't need to throw IOException
        Hide
        Uwe Schindler added a comment -

        Thanks!
        I think if some code throws IOExceptions in setReader() it is doing something wrong. I would suggest to move the code doing this to reset(). Almost all Tokenizers are doing this from there.

        Show
        Uwe Schindler added a comment - Thanks! I think if some code throws IOExceptions in setReader() it is doing something wrong. I would suggest to move the code doing this to reset(). Almost all Tokenizers are doing this from there.
        Hide
        ASF subversion and git services added a comment -

        Commit 1712735 from Alan Woodward in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1712735 ]

        LUCENE-6884: Analyzer.tokenStream() and Tokenizer.setReader() don't need to throw IOException

        Show
        ASF subversion and git services added a comment - Commit 1712735 from Alan Woodward in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1712735 ] LUCENE-6884 : Analyzer.tokenStream() and Tokenizer.setReader() don't need to throw IOException

          People

          • Assignee:
            Alan Woodward
            Reporter:
            Alan Woodward
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development