Solr
  1. Solr
  2. SOLR-7697

Schema API doesn't take class or luceneMatchVersion attributes into account for the analyzer when adding a new field type

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 5.2
    • Fix Version/s: 5.3, 6.0
    • Component/s: Schema and Analysis
    • Labels:
      None

      Description

      I've noticed that in schema.xml can be defined field types like the following :

          <fieldType name="nametext" class="solr.TextField">
            <analyzer class="org.apache.lucene.analysis.core.WhitespaceAnalyzer"/>
          </fieldType>
      

      The current add-field-type update operation from Schema API doesn't take into account the class (and the luceneMatchVersion - see FieldTypePluginLoader.java) for the field type analyzer definition, but only their type.

      See FieldTypeXmlAdapter.java

        protected static Element createAnalyzerElement(Document doc, String type, Map<String,?> json) {
          Element analyzer = doc.createElement("analyzer");
          if (type != null)
            analyzer.setAttribute("type", type);
            .....
      

      If the change would be made, the add-field-type request would look like this :

      curl -X POST -H 'Content-type:application/json' --data-binary '{
        "add-field-type": {
          "name": "nametext",
          "class": "solr.TextField",
          "analyzer": {
            "class": "org.apache.lucene.analysis.core.WhitespaceAnalyzer"
          }
        }
      }' http://localhost:8983/solr/gettingstarted/schema
      
      1. SOLR-7697.patch
        10 kB
        Steve Rowe
      2. SOLR-7697.patch
        4 kB
        Marius Grama

        Issue Links

          Activity

          Hide
          Marius Grama added a comment -

          Here is a patch along with a unit test for the reported issue.

          Show
          Marius Grama added a comment - Here is a patch along with a unit test for the reported issue.
          Hide
          Steve Rowe added a comment -

          Thanks Marius.

          I'm attaching a modified patch:

          1. Specifying an analyzer class on a field type is mutually exclusive of specification of char filters, a tokenizer, and filters - I modified FieldTypeXmlAdaptor.createAnalyzerElement() to throw exceptions for requests that improperly mix these, and added a test confirming that to TestBulkSchemaAPI.
          2. FieldType.getAnalyzerProperties() wasn't outputting luceneMatchVersion, so although field type addition/modification was properly accepting specification of luceneMatchVersion when an analyzer class was specified, serialization wasn't preserving the information. I fixed it, and added a test that the round-tripping works to TestBulkSchemaAPI.
          3. Your patch moved some imports around in FieldTypeXmlAdaptor but didn't otherwise change them. In general, don't reorder imports unless they're completely unordered or you change them significantly. You may have to change a setting in your IDE to address this. (I put them back in my patch.)

          I think it's ready to go - I'll commit shortly.

          Show
          Steve Rowe added a comment - Thanks Marius. I'm attaching a modified patch: Specifying an analyzer class on a field type is mutually exclusive of specification of char filters, a tokenizer, and filters - I modified FieldTypeXmlAdaptor.createAnalyzerElement() to throw exceptions for requests that improperly mix these, and added a test confirming that to TestBulkSchemaAPI . FieldType.getAnalyzerProperties() wasn't outputting luceneMatchVersion , so although field type addition/modification was properly accepting specification of luceneMatchVersion when an analyzer class was specified, serialization wasn't preserving the information. I fixed it, and added a test that the round-tripping works to TestBulkSchemaAPI . Your patch moved some imports around in FieldTypeXmlAdaptor but didn't otherwise change them. In general, don't reorder imports unless they're completely unordered or you change them significantly. You may have to change a setting in your IDE to address this. (I put them back in my patch.) I think it's ready to go - I'll commit shortly.
          Hide
          Marius Grama added a comment -

          Steve Rowe I just had a look over your patch. It is definitely an improvement to my initial version. FieldTypeXmlAdapter provides now meaningful exception messages.

          It looks good to me.

          Show
          Marius Grama added a comment - Steve Rowe I just had a look over your patch. It is definitely an improvement to my initial version. FieldTypeXmlAdapter provides now meaningful exception messages. It looks good to me.
          Hide
          ASF subversion and git services added a comment -

          Commit 1686327 from Steve Rowe in branch 'dev/trunk'
          [ https://svn.apache.org/r1686327 ]

          SOLR-7697: Schema API doesn't take class or luceneMatchVersion attributes into account for the analyzer when adding a new field type.

          Show
          ASF subversion and git services added a comment - Commit 1686327 from Steve Rowe in branch 'dev/trunk' [ https://svn.apache.org/r1686327 ] SOLR-7697 : Schema API doesn't take class or luceneMatchVersion attributes into account for the analyzer when adding a new field type.
          Hide
          ASF subversion and git services added a comment -

          Commit 1686329 from Steve Rowe in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1686329 ]

          SOLR-7697: Schema API doesn't take class or luceneMatchVersion attributes into account for the analyzer when adding a new field type. (merged trunk r1686327)

          Show
          ASF subversion and git services added a comment - Commit 1686329 from Steve Rowe in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1686329 ] SOLR-7697 : Schema API doesn't take class or luceneMatchVersion attributes into account for the analyzer when adding a new field type. (merged trunk r1686327)
          Hide
          Steve Rowe added a comment -

          Committed to trunk and branch_5x.

          Thanks Marius!

          Show
          Steve Rowe added a comment - Committed to trunk and branch_5x. Thanks Marius!
          Hide
          Shalin Shekhar Mangar added a comment -

          Bulk close for 5.3.0 release

          Show
          Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

            People

            • Assignee:
              Steve Rowe
              Reporter:
              Marius Grama
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development