Solr
  1. Solr
  2. SOLR-309

A solr.StrField that has analyzers configured should emit warning to log

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: search
    • Labels:
      None

      Description

      A solr.StrField that has analyzers configured in schema.xml should emit a warning/error to the log. As I understand StrFields never get tokenized.

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          personally, i'd vote for changing the FieldType loading code to allow analyzers for any field type (instead of the special logic that currently only looks for them in the case of TextField) ... even if i have an instance of SortableIntFieldType i should be able to put an analyzer in front of it that will massage the input data into something thta parses cleanly as an integer.

          Show
          Hoss Man added a comment - personally, i'd vote for changing the FieldType loading code to allow analyzers for any field type (instead of the special logic that currently only looks for them in the case of TextField) ... even if i have an instance of SortableIntFieldType i should be able to put an analyzer in front of it that will massage the input data into something thta parses cleanly as an integer.
          Hide
          Andy Blower added a comment -

          Has this been changed in the latest version of SOLR? I'm using v1.2.0 and it was unclear from the documentation whether it is valid to specify analyzers for non-text fields. I tried it when I wanted a simple tokenized string field, and it appears to work. Here's my field definition:

          <!-- Fieldtype for simple tokenized strings. (Don't use for sorting, do use for space separated codes) -->
          <fieldType name="stringText" class="solr.TextField" positionIncrementGap="0" omitNorms="true">
          <analyzer>
          <tokenizer class="solr.WhitespaceTokenizerFactory"/>
          </analyzer>
          </fieldType>

          I use it for fields that have whitespace separated codes e.g. "101 104 15a 075". Just using a string field meant I could only find the exact full string rather than individual codes.

          Show
          Andy Blower added a comment - Has this been changed in the latest version of SOLR? I'm using v1.2.0 and it was unclear from the documentation whether it is valid to specify analyzers for non-text fields. I tried it when I wanted a simple tokenized string field, and it appears to work. Here's my field definition: <!-- Fieldtype for simple tokenized strings. (Don't use for sorting, do use for space separated codes) --> <fieldType name="stringText" class="solr.TextField" positionIncrementGap="0" omitNorms="true"> <analyzer> <tokenizer class="solr.WhitespaceTokenizerFactory"/> </analyzer> </fieldType> I use it for fields that have whitespace separated codes e.g. "101 104 15a 075". Just using a string field meant I could only find the exact full string rather than individual codes.
          Hide
          Yonik Seeley added a comment -

          > I tried it when I wanted a simple tokenized string field

          The analyzer you show is a text field (which is what you want when doing analysis).
          Why would you want an "analyzed" string field when that is exactly what a text field is?

          Show
          Yonik Seeley added a comment - > I tried it when I wanted a simple tokenized string field The analyzer you show is a text field (which is what you want when doing analysis). Why would you want an "analyzed" string field when that is exactly what a text field is?
          Hide
          Hoss Man added a comment -

          This came up in IRC today, after a lot of back and forth there was a proposal to change FieldType.setAnalyzer (and setQueryAnalyzer) so that by default fieldtypes throw an exception if use try to set ana analyzer on them

          this should fail quickly if people mistakenly put an analyzer on a FieldType (in schema.xml) that doesn't support it.

          only TextField will allow setting an analyzer (since TextField is currently the only class that uses it)

          Show
          Hoss Man added a comment - This came up in IRC today, after a lot of back and forth there was a proposal to change FieldType.setAnalyzer (and setQueryAnalyzer) so that by default fieldtypes throw an exception if use try to set ana analyzer on them this should fail quickly if people mistakenly put an analyzer on a FieldType (in schema.xml) that doesn't support it. only TextField will allow setting an analyzer (since TextField is currently the only class that uses it)
          Hide
          Hoss Man added a comment -

          going to try and get a fix for this into 3.1

          Show
          Hoss Man added a comment - going to try and get a fix for this into 3.1
          Hide
          Hoss Man added a comment -

          path that modifies FieldType.setAnalyzer to fail by default, so that only TextField allows the specification of an analyzer.

          this leaves the door open for other future FieldTypes to also allow analyzers, but solve the problem of Solr silently allowing analyzers to be set on field types that don't actually use them

          (and since the bogus analyzer can no longer be set on other field types, analysis.jsp won't use that bogus analyzer)

          Show
          Hoss Man added a comment - path that modifies FieldType.setAnalyzer to fail by default, so that only TextField allows the specification of an analyzer. this leaves the door open for other future FieldTypes to also allow analyzers, but solve the problem of Solr silently allowing analyzers to be set on field types that don't actually use them (and since the bogus analyzer can no longer be set on other field types, analysis.jsp won't use that bogus analyzer)
          Hide
          Hoss Man added a comment -

          Committed revision 1076432. - trunk
          Committed revision 1076444. - 3x

          Show
          Hoss Man added a comment - Committed revision 1076432. - trunk Committed revision 1076444. - 3x
          Hide
          Grant Ingersoll added a comment -

          Bulk close for 3.1.0 release

          Show
          Grant Ingersoll added a comment - Bulk close for 3.1.0 release

            People

            • Assignee:
              Hoss Man
              Reporter:
              Thomas Peuss
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development