Solr
  1. Solr
  2. SOLR-2474

Analysis.jsp and AnalaysisRequestHandlerBase do not correctly clear attributes on caching tokens

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 3.2, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      When caching tokens, the helper TokenStreams in analysis.jsp and AnalysisRequestHandlerBase do not clear all attributes.
      The issue is tricky. The cached tokens do not contain all attributes in early stages, so copyTo() does not necessarily overwrite all attributes in "this". Calling clearAttributes ensures this.

      Was: LUCENE-2901 broke protected words by only setting and never clearing (that change should have been accompanied by offsetting code to clear the attribute somewhere).

      The problem here was, that this attribute was added later in anaylsis chain, so cached tokens don't include this. Sorry, that was my fault when rewriting analysis.jsp together with Robert

      1. SOLR-2474.patch
        1 kB
        Uwe Schindler

        Activity

        Robert Muir made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Uwe Schindler made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Fix Version/s 3.2 [ 12316172 ]
        Fix Version/s 4.0 [ 12314992 ]
        Resolution Fixed [ 1 ]
        Uwe Schindler made changes -
        Resolution Invalid [ 6 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Uwe Schindler made changes -
        Summary Aanalysis.jsp and AnalaysisRequestHandlerBase does not correctly clear attributes on caching tokens Analysis.jsp and AnalaysisRequestHandlerBase do not correctly clear attributes on caching tokens
        Description LUCENE-2901 broke protected words by only setting and never clearing (that change should have been accompanied by offsetting code to clear the attribute somewhere). When caching tokens, the helper TokenStreams in analysis.jsp and AnalysisRequestHandlerBase do not clear all attributes.
        The issue is tricky. The cached tokens do not contain all attributes in early stages, so copyTo() does not necessarily overwrite all attributes in "this". Calling clearAttributes ensures this.

        bq. Was: LUCENE-2901 broke protected words by only setting and never clearing (that change should have been accompanied by offsetting code to clear the attribute somewhere).

        The problem here was, that this attribute was added later in anaylsis chain, so cached tokens don't include this. Sorry, that was my fault when rewriting analysis.jsp together with Robert :(
        Uwe Schindler made changes -
        Attachment SOLR-2474.patch [ 12476931 ]
        Uwe Schindler made changes -
        Assignee Uwe Schindler [ thetaphi ]
        Uwe Schindler made changes -
        Summary all words are protected after the first detected Aanalysis.jsp and AnalaysisRequestHandlerBase does not correctly clear attributes on caching tokens
        Uwe Schindler made changes -
        Affects Version/s 3.1 [ 12314371 ]
        Affects Version/s 3.1 [ 12314822 ]
        Project Lucene - Java [ 12310110 ] Solr [ 12310230 ]
        Key LUCENE-3039 SOLR-2474
        Lucene Fields [New]
        Robert Muir made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Invalid [ 6 ]
        Yonik Seeley made changes -
        Field Original Value New Value
        Affects Version/s 3.1 [ 12314822 ]
        Yonik Seeley created issue -

          People

          • Assignee:
            Uwe Schindler
            Reporter:
            Yonik Seeley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development