[TIKA-1133] Ability to Allow Empty and Duplicate Tika Values for XML Elements

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3
• Fix Version/s: 1.4
• Component/s: parser
• Assignee: Ray Gauss II
• Reporter: Ray Gauss II
• Created: 11/Jun/13 02:59
• Updated: 11/Jun/13 03:17
• Resolved: 11/Jun/13 03:17

Description

In some cases it is beneficial to allow empty and duplicate Tika metadata values for multi-valued XML elements like RDF bags.

Consider an example where the original source metadata is structured something like:

<Person>
  <FirstName>John</FirstName>
  <LastName>Smith</FirstName>
</Person>
<Person>
  <FirstName>Jane</FirstName>
  <LastName>Doe</FirstName>
</Person>
<Person>
  <FirstName>Bob</FirstName>
</Person>
<Person>
  <FirstName>Kate</FirstName>
  <LastName>Smith</FirstName>
</Person>

and since Tika stores only flat metadata we transform that before invoking a parser to something like:

 <custom:FirstName>
  <rdf:Bag>
   <rdf:li>John</rdf:li>
   <rdf:li>Jane</rdf:li>
   <rdf:li>Bob</rdf:li>
   <rdf:li>Kate</rdf:li>
  </rdf:Bag>
 </custom:FirstName>
 <custom:LastName>
  <rdf:Bag>
   <rdf:li>Smith</rdf:li>
   <rdf:li>Doe</rdf:li>
   <rdf:li></rdf:li>
   <rdf:li>Smith</rdf:li>
  </rdf:Bag>
 </custom:LastName>

The current behavior ignores empties and duplicates and we don't know if Bob or Kate ever had last names. Empties or duplicates in other positions result in an incorrect mapping of data.

We should allow the option to create an ElementMetadataHandler which allows empty and/or duplicate values.