Solr
  1. Solr
  2. SOLR-4567

copyField source glob matching explicit field(s) stopped working in Solr 4.2

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.2
    • Fix Version/s: 4.2.1, 4.3, 6.0
    • Component/s: Schema and Analysis
    • Labels:
      None

      Description

      Alexandre Rafalovitch reported on the solr-user mailing list http://markmail.org/message/sqhzxaet55wztldq that a copyField directive that worked under Solr 4.1 has stopped working under Solr 4.2:

      <field name="addr_from" type="email" indexed="true" stored="true" required="true" />
      <field name="addr_to" type="email" multiValued="true" indexed="true" stored="true" required="true" />
      <copyField source="addr_*" dest="text" />
      

      I'm 99% sure that matching multiple explicit fields is not the problem, but rather the issue is that I committed changes under SOLR-4503 to handle additional previously unhandled changes, but I didn't consider this case, and so introduced the regression. The table on SOLR-3798 that attempted to describe all possible combinations of copyField directives does not include this case.

      UPDATE: Workaround: instead of using a single copyField directive matching multiple explicit source fields, like the above example, use multiple copyField directives, each giving an explicit field name. So the above example using this workaround would be:

      <field name="addr_from" type="email" indexed="true" stored="true" required="true" />
      <field name="addr_to" type="email" multiValued="true" indexed="true" stored="true" required="true" />
      <copyField source="addr_from" dest="text" />
      <copyField source="addr_to" dest="text" />
      

        Issue Links

          Activity

          Hide
          Steve Rowe added a comment -

          Here's a version, as of Solr 4.2, of the SOLR-3798 table enumerating all possible combinations of copyField source and dest value types, expanded to include Alex's case: copyField source pattern matching explicit field(s) - note that destination patterns matching explicit field(s) are excluded because they are nonsensical: the asterisk could create field names that wouldn't be the same as the explicit field.

          The following declarations are matched by the examples in the table below:

          <field name="text" .../>
          <field name="title" .../>
          <field name="text_en" .../>
          <dynamicField name="*_i" .../>
          <dynamicField name="*_s" .../>
          
          case # source value type dest value type Example Schema parse succeeds?
          1 <field> name <field> name <copyField source="title" dest="text"/> Yes
          2 <field> name <dynamicField> name <copyField source="title" dest="*_s"/> No: "copyField only supports a dynamic destination if the source is also dynamic"
          3 <field> name subset pattern[1] <copyField source="title" dest="*_dest_sub_s"/> No: "copyField only supports a dynamic destination if the source is also dynamic"
          4 <field> name subset pattern no asterisk[2] <copyField source="title" dest="dest_sub_no_ast_s"/> Yes
           
          5 <dynamicField> name <field> name <copyField source="*_i" dest="title"/> Yes
          6 <dynamicField> name <dynamicField> name <copyField source="*_i" dest="*_s"/> Yes
          7 <dynamicField> name subset pattern <copyField source="*_i" dest="*_dest_sub_s"/> Yes
          8 <dynamicField> name subset pattern no asterisk <copyField source="*_i" dest="dest_sub_no_ast_s"/> Yes
           
          9 subset pattern[1] <field> name <copyField source="*_src_sub_i" dest="title"/> Yes
          10 subset pattern <dynamicField> name <copyField source="*_src_sub_i" dest="*_s"/> Yes
          11 subset pattern subset pattern <copyField source="*_src_sub_i" dest="*_dest_sub_s"/> Yes
          12 subset pattern subset pattern no asterisk <copyField source="*_src_sub_i" dest="dest_sub_no_ast_s"/> Yes
           
          13 subset pattern no asterisk[2] <field> name <copyField source="src_sub_no_ast_i" dest="title"/> Yes
          14 subset pattern no asterisk <dynamicField> name <copyField source="src_sub_no_ast_i" dest="*_s"/> Yes
          15 subset pattern no asterisk subset pattern <copyField source="src_sub_no_ast_i" dest="*_dest_sub_s"/> Yes
          16 subset pattern no asterisk subset pattern no asterisk <copyField source="src_sub_no_ast_i" dest="dest_sub_no_ast_s"/> Yes
           
          17 pattern matching explicit field(s) <field> name <copyField source="text_*" dest="text"/> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField."
          18 pattern matching explicit field(s) <dynamicField> name <copyField source="text_*" dest="*_s"/> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField."
          19 pattern matching explicit field(s) subset pattern <copyField source="text_*" dest="*_dest_sub_s"/> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField."
          20 pattern matching explicit field(s) subset pattern no asterisk <copyField source="text_*" dest="dest_sub_no_ast_s"/> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField."

          [1] A "subset pattern" matches a subset of the language recognized by a declared dynamic field's pattern
          [2] A "subset pattern no asterisk" is one member from the language recognized by a declared dynamic field's pattern

          Show
          Steve Rowe added a comment - Here's a version, as of Solr 4.2, of the SOLR-3798 table enumerating all possible combinations of copyField source and dest value types, expanded to include Alex's case: copyField source pattern matching explicit field(s) - note that destination patterns matching explicit field(s) are excluded because they are nonsensical: the asterisk could create field names that wouldn't be the same as the explicit field. The following declarations are matched by the examples in the table below: <field name= "text" .../> <field name= "title" .../> <field name= "text_en" .../> <dynamicField name= "*_i" .../> <dynamicField name= "*_s" .../> case # source value type dest value type Example Schema parse succeeds? 1 <field> name <field> name <copyField source="title" dest="text" /> Yes 2 <field> name <dynamicField> name <copyField source="title" dest="*_s" /> No: "copyField only supports a dynamic destination if the source is also dynamic" 3 <field> name subset pattern [1] <copyField source="title" dest="*_dest_sub_s" /> No: "copyField only supports a dynamic destination if the source is also dynamic" 4 <field> name subset pattern no asterisk [2] <copyField source="title" dest="dest_sub_no_ast_s" /> Yes   5 <dynamicField> name <field> name <copyField source="*_i" dest="title" /> Yes 6 <dynamicField> name <dynamicField> name <copyField source="*_i" dest="*_s" /> Yes 7 <dynamicField> name subset pattern <copyField source="*_i" dest="*_dest_sub_s" /> Yes 8 <dynamicField> name subset pattern no asterisk <copyField source="*_i" dest="dest_sub_no_ast_s" /> Yes   9 subset pattern [1] <field> name <copyField source="*_src_sub_i" dest="title" /> Yes 10 subset pattern <dynamicField> name <copyField source="*_src_sub_i" dest="*_s" /> Yes 11 subset pattern subset pattern <copyField source="*_src_sub_i" dest="*_dest_sub_s" /> Yes 12 subset pattern subset pattern no asterisk <copyField source="*_src_sub_i" dest="dest_sub_no_ast_s" /> Yes   13 subset pattern no asterisk [2] <field> name <copyField source="src_sub_no_ast_i" dest="title" /> Yes 14 subset pattern no asterisk <dynamicField> name <copyField source="src_sub_no_ast_i" dest="*_s" /> Yes 15 subset pattern no asterisk subset pattern <copyField source="src_sub_no_ast_i" dest="*_dest_sub_s" /> Yes 16 subset pattern no asterisk subset pattern no asterisk <copyField source="src_sub_no_ast_i" dest="dest_sub_no_ast_s" /> Yes   17 pattern matching explicit field(s) <field> name <copyField source="text_*" dest="text" /> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField." 18 pattern matching explicit field(s) <dynamicField> name <copyField source="text_*" dest="*_s" /> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField." 19 pattern matching explicit field(s) subset pattern <copyField source="text_*" dest="*_dest_sub_s" /> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField." 20 pattern matching explicit field(s) subset pattern no asterisk <copyField source="text_*" dest="dest_sub_no_ast_s" /> No: "copyField source :'text_*' is not an explicit field and doesn't match a dynamicField." [1] A "subset pattern" matches a subset of the language recognized by a declared dynamic field's pattern [2] A "subset pattern no asterisk" is one member from the language recognized by a declared dynamic field's pattern
          Hide
          Steve Rowe added a comment -

          Patch fixing the issue - when the examples for cases 17-20 from the above table are added to schema-copyfield-test.xml without the rest of this patch, CopyFieldTest fails because schema parsing fails. With the rest of the patch, schema parsing succeeds for all 4 cases.

          Patch also contains modifications to the REST API /solr/schema/copyfields and its tests, so that the matching source explicit fields (if any) are included in the response.

          Committing shortly.

          Show
          Steve Rowe added a comment - Patch fixing the issue - when the examples for cases 17-20 from the above table are added to schema-copyfield-test.xml without the rest of this patch, CopyFieldTest fails because schema parsing fails. With the rest of the patch, schema parsing succeeds for all 4 cases. Patch also contains modifications to the REST API /solr/schema/copyfields and its tests, so that the matching source explicit fields (if any) are included in the response. Committing shortly.
          Hide
          Steve Rowe added a comment -

          Committed to trunk r1455832 and branch_4x r1455837.

          Thanks Alex for reporting the bug!

          Show
          Steve Rowe added a comment - Committed to trunk r1455832 and branch_4x r1455837. Thanks Alex for reporting the bug!
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1455837

          SOLR-4567: copyField source glob matching explicit field(s) stopped working in Solr 4.2 (merged trunk r1455832)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1455837 SOLR-4567 : copyField source glob matching explicit field(s) stopped working in Solr 4.2 (merged trunk r1455832)
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1455832

          SOLR-4567: copyField source glob matching explicit field(s) stopped working in Solr 4.2

          Show
          Commit Tag Bot added a comment - [trunk commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1455832 SOLR-4567 : copyField source glob matching explicit field(s) stopped working in Solr 4.2
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1456288

          SOLR-4567: Beef up copyField tests, and assert schema conforms to test assumptions

          Show
          Commit Tag Bot added a comment - [trunk commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1456288 SOLR-4567 : Beef up copyField tests, and assert schema conforms to test assumptions
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1456289

          SOLR-4567: Beef up copyField tests, and assert schema conforms to test assumptions (merged trunk r1456288)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1456289 SOLR-4567 : Beef up copyField tests, and assert schema conforms to test assumptions (merged trunk r1456288)
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1457630

          Move SOLR-4567 from 4.3 to 4.2.1 (merged trunk r1457629)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1457630 Move SOLR-4567 from 4.3 to 4.2.1 (merged trunk r1457629)
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Steven Rowe
          http://svn.apache.org/viewvc?view=revision&revision=1457629

          Move SOLR-4567 from 4.3 to 4.2.1

          Show
          Commit Tag Bot added a comment - [trunk commit] Steven Rowe http://svn.apache.org/viewvc?view=revision&revision=1457629 Move SOLR-4567 from 4.3 to 4.2.1
          Hide
          Uwe Schindler added a comment -

          Closed after release.

          Show
          Uwe Schindler added a comment - Closed after release.

            People

            • Assignee:
              Steve Rowe
              Reporter:
              Steve Rowe
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development