Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10218

The Schema API commands "add-field-type" and "replace-field-type" improperly specify SimilarityFactory params

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 6.4.1
    • Fix Version/s: 6.5, 7.0
    • Component/s: Schema and Analysis
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      When sending a JSON POST to the Schema API to replace a field type, the following JSON does not pass the SolrParameters properly to the BM25SimilarityFactory.

      {"replace-field-type":{"name":"tint","class":"solr.TrieIntField","positionIncrementGap":"0","precisionStep":"8","similarity":{"class":"solr.BM25SimilarityFactory","k1":1.25,"b":0.75}}}
      

      The `appendAttrs` function in the FieldTypeXmlAdapter parses k1 and b into attributes instead of children. https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/rest/schema/FieldTypeXmlAdapter.java#L155

      <similarity b="0.75" class="org.apache.lucene.search.similarities.BM25Similarity" k1="1.25"/>
      

      Based on the XML examples for similarity, this should actually be the following :

      <similarity class="org.apache.lucene.search.similarities.BM25Similarity">
           <float name=“k1”>0.1</float>
           <float name=“b”>0.1</float>
      </similarity>
      

      The similarities block in JSON should be handled differently so that the XML is generated appropriately.

          protected static Element appendSimilarityAttrs(Document doc, Element elm, Map<String,?> json) {
              String clazz = (String) json.get("class");
              elm.setAttribute("class", clazz);
              json.remove("class");
      
              for (Map.Entry<String,?> entry : json.entrySet()) {
                  Object val = entry.getValue();
                  if (val != null && !(val instanceof Map)) {
                      Element element = doc.createElement(val.getClass().getSimpleName().toLowerCase());
                      element.setAttribute("name", entry.getKey());
                      element.setTextContent(entry.getValue().toString());
                      elm.appendChild(element);
                  }
              }
              return elm;
          }
      
      1. SOLR-10218.patch
        8 kB
        Steve Rowe
      2. SOLR-10218.patch
        5 kB
        Troy Mohl

        Activity

        Hide
        troymohl Troy Mohl added a comment -

        This patch updates the FieldTypeXmlAdapter to parse the similarity options into XML elements.

        Show
        troymohl Troy Mohl added a comment - This patch updates the FieldTypeXmlAdapter to parse the similarity options into XML elements.
        Hide
        steve_rowe Steve Rowe added a comment -

        Modified patch, I think it's ready to go. Changes:

        • Boolean and String params are now properly specified (the XML tags are "bool" and "str" for these, respectively).
        • Fixed removal of "class" param prior to param addition.
        • Added testing of replace-field-type command.
        • Added testing of another similarity, with Boolean and String params: DFISimilarity.
        • Refactored some test code to reuse an existing utility method and eliminate an added one.

        I'll commit this, including to the 6.5 release branch, once all tests pass for me.

        Show
        steve_rowe Steve Rowe added a comment - Modified patch, I think it's ready to go. Changes: Boolean and String params are now properly specified (the XML tags are "bool" and "str" for these, respectively). Fixed removal of "class" param prior to param addition. Added testing of replace-field-type command. Added testing of another similarity, with Boolean and String params: DFISimilarity. Refactored some test code to reuse an existing utility method and eliminate an added one. I'll commit this, including to the 6.5 release branch, once all tests pass for me.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2e4b994f71022989f7c9695a91774029261b6922 in lucene-solr's branch refs/heads/branch_6_5 from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2e4b994 ]

        SOLR-10218: The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2e4b994f71022989f7c9695a91774029261b6922 in lucene-solr's branch refs/heads/branch_6_5 from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2e4b994 ] SOLR-10218 : The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ef141de630e03a7e8725f6b4a8d11e35f14d41c2 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ef141de ]

        SOLR-10218: The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params

        Show
        jira-bot ASF subversion and git services added a comment - Commit ef141de630e03a7e8725f6b4a8d11e35f14d41c2 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ef141de ] SOLR-10218 : The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9b57545df01fd0f20fd473e80d4fba621a51abf9 in lucene-solr's branch refs/heads/master from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9b57545 ]

        SOLR-10218: The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9b57545df01fd0f20fd473e80d4fba621a51abf9 in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9b57545 ] SOLR-10218 : The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params
        Hide
        steve_rowe Steve Rowe added a comment -

        Committed to master, branch_6x and branch_6_5 (for inclusion in Solr 6.5).

        Thanks Benjamin Deininger and Troy Mohl !

        Show
        steve_rowe Steve Rowe added a comment - Committed to master, branch_6x and branch_6_5 (for inclusion in Solr 6.5). Thanks Benjamin Deininger and Troy Mohl !

          People

          • Assignee:
            steve_rowe Steve Rowe
            Reporter:
            bdeininger Benjamin Deininger
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development