[TIKA-1607] Introduce new arbitrary object key/values data structure for persistence of Tika Metadata

• Type: Sub-task
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Fix Version/s: 1.17, 2.0.0-BETA, 2.1.0
• Component/s: core, metadata
• Assignee: Lewis John McGibbney
• Reporter: Lewis John McGibbney
• Created: 20/Apr/15 21:44
• Updated: 17/Aug/21 13:30

Description

I am currently working implementing more comprehensive extraction and enhancement of the Tika support for Phone number extraction and metadata modeling.
Right now we utilize the String[] multivalued support available within Tika to persist phone numbers as

Metadata: String: String[]
Metadata: phonenumbers: number1, number2, number3, ...

I would like to propose we extend multi-valued support outside of the String[] paradigm by implementing a more abstract Collection of Objects such that we could consider and implement the phone number use case as follows

Metadata: String:  Object

Where Object could be a Collection<HashMap<String/Property, HashMap<String/Property, String/Int/Long>> e.g.

Metadata: phonenumbers: [([phone]: (LibPN-CountryCode : US), (LibPN-NumberType: International), (etc: etc)...), ([phone]: LibPN-CountryCode : UK), (LibPN-NumberType: International), (etc: etc)...) (etc)] 

There are obvious backwards compatibility issues with this approach... additionally it is a fundamental change to the code Metadata API. I hope that the <String, Object> Mapping however is flexible enough to allow me to model Tika Metadata the way I want.

Any comments folks? Thanks
Lewis

Issue Links

• blocks: TIKA-1609 Leverage Google's LibPhonenumber for enhanced phone number extraction and metadata modeling
• is depended upon by: TIKA-1295 Make some Dublin Core items multi-valued
• is related to: TIKA-1691 Apache Tika for enabling metadata interoperability
• is related to: TIKA-1857 Enhance PDFParser to extract text from XFA forms