[SOLR-1973] Empty fields in update messages confuse DataImportHandler

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4, 1.4.1
• Fix Version/s: 3.1, 4.0-ALPHA
• Component/s: contrib - DataImportHandler

Environment

CentOS 5, Java 1.6, Tomcat 6

• Assignee: Koji Sekiguchi
• Reporter: Sixten Otto
• Created: 25/Jun/10 05:34
• Updated: 30/Mar/11 15:45
• Resolved: 08/Nov/10 01:38

Description

I seem to be running into an issue with Solr (maybe just the DataImportHandler?) not liking empty field elements in the docs, and getting the wrong values into the fields of the index. Here's the entity declaration from data-config.xml for my isolated example:

<document>
<entity name="contentAsSolrAdd"
dataSource="xml"
processor="XPathEntityProcessor"
stream="true"
url="http://example.com/Content.xml"
useSolrAddSchema="true">
</entity>
</document>

And here's the Content.xml being pulled in by the DIH:

<add>
<doc>
<field name="empty"></field>
<field name="full">Lorem Ipsum Dolor</field>
<field name="other">Some content is me!</field>
</doc>
</add>

And here's the relevant portion of the output from the DIH in debug mode:

<lst name="document#1">
<str name="query">
http://example.com/Content.xml
</str>
<str name="time-taken">0:0:0.6</str>
<str>----------- row #1-------------</str>
<str name="full">Some content is me!</str>
<str name="empty">Lorem Ipsum Dolor</str>
<str>---------------------------------------------</str>
</lst>

Notice that the field "full" doesn't appear here, but the following field "empty" has the content that was there for "full". The "other" field, which was non-empty, and preceded by a non-empty field, shows up correctly.