Solr
  1. Solr
  2. SOLR-2039

Multivalued fields with dynamic names does not work properly with DIH

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Windows XP, Default Solr 1.4 install with jetty

      Description

      Attempting to use multiValued fields using the DataImportHandler with dynamic names does not seem to work as intended.

      Setting up the following in schema.xml
      <dynamicField name="*_s" type="string" indexed="true" stored="true" multiValued="true" />

      Then attempting to import a multiValued field through a RDBMS using children records with a dynamic name. The snippet from data-config.xml

      <entity name="terms" query="select distinct DESC_TERM from tem_metadata where item_id=$

      {item.ID_PK}">
      <entity name="metadata" query="select * from term_metadata where item_id=${item.ID_PK}

      AND desc_term='$

      {terms.DESC_TERM}'" >
      <field name="${terms.DESC_TERM}

      _s" column="TEXT_VALUE" />
      </entity>
      </entity>

      Results in only the FIRST record being imported. If we change the field name in the above example to a constant (ie above field entry becomes: <field name="metadata_record_s" column="TEXT_VALUE" />). The multiple records are correctly imported.

      This was posted on solr-user, and others have reported the problem on the mailing list archive.

      The workaround was to use a javascript transform to perform the same behavior. Changes in data-config.xml become:

      <entity name="metadata" query="select * from vw_item_metadata where item_id=$

      {item.DIVID_PK}

      AND core_desc_term='$

      {terms.CORE_DESC_TERM}

      '" transformer="script:f1"/>

      <script><![CDATA[
      function f1(row)

      { var name = row.get('CORE_DESC_TERM'); var value = row.get('TEXT_VALUE') row.put(name+ '_s', value); return row; }

      ]]></script>

      This results in multivalued field with a dynamic name assigned.

      1. SOLR-2039.patch
        0.8 kB
        ruslan.shv

        Activity

        Hide
        Chris Male added a comment -

        +1 to fixing this. I have also seen this bug in the wild.

        Show
        Chris Male added a comment - +1 to fixing this. I have also seen this bug in the wild.
        Hide
        Koji Sekiguchi added a comment -

        Just assigning a component

        Show
        Koji Sekiguchi added a comment - Just assigning a component
        Hide
        ruslan.shv added a comment -

        how i understand, the issue is that DIH cannot import dynamic fields with specific field names. for example, i have following
        schema.xml

        <dynamicField name="*_s" type="string" indexed="true" stored="true" multiValued="true" />

        data-config.xml

        <entity name="items" query="select * from item">
        <field name="${items.item_name}_s" column="item_value" />
        </entity>

        DB with following content

        item_name item_value
        item1 value1
        item2 value2

        and i want to Solr indexed my data such way, that i would get something like this

        <doc>
        <str name="item1_s">
        value1
        </str>
        <str name="item2_s">
        value2
        </str>
        </doc>

        Show
        ruslan.shv added a comment - how i understand, the issue is that DIH cannot import dynamic fields with specific field names. for example, i have following schema.xml <dynamicField name="*_s" type="string" indexed="true" stored="true" multiValued="true" /> data-config.xml <entity name="items" query="select * from item"> <field name="${items.item_name}_s" column="item_value" /> </entity> DB with following content item_name item_value item1 value1 item2 value2 and i want to Solr indexed my data such way, that i would get something like this <doc> <str name="item1_s"> value1 </str> <str name="item2_s"> value2 </str> </doc>
        Hide
        ruslan.shv added a comment -

        I hope this patch will help solve the issue

        Show
        ruslan.shv added a comment - I hope this patch will help solve the issue

          People

          • Assignee:
            Unassigned
            Reporter:
            K A
          • Votes:
            5 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:

              Development