[CONNECTORS-984] Give Tika's metadata some hints

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 1.7
• Fix Version/s: ManifoldCF 1.7
• Component/s: Amazon CloudSearch output connector
• Assignee: Shinichiro Abe
• Reporter: Shinichiro Abe
• Created: 24/Jun/14 14:57
• Updated: 25/Jul/14 05:33
• Resolved: 23/Jul/14 11:51

Description

Component: Tika connector

Currently in trunk code, we don't set data in Tika's metadata object.
We likely have to give metadata some hints to detect and extract from document.

• resourceName
• ContentType
• stream size
• charset(new feature)
• Password handling(new feature)

Also, when TikaException(e.g. parsing error at PDFBox/POI) is thrown, we need to decide to ignore or not about the parsing document. Solr Cell has 'ignoreTikaException' param. When TikaException is thrown, if true, metadata only is indexed, if false, Solr responds server error and the document is not indexed.

Reference-->Solr Cell:
http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ExtractingDocumentLoader.java?view=markup#l142