[NIFI-4514] Import Solr schema to NiFi record schema

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Koji Kawamura
• Created: 23/Oct/17 09:40
• Updated: 23/Oct/17 09:40

Description

As Solr related components start supporting NiFi record data model, it would be helpful if NiFi Solr processors can create NiFi record schema from Solr document schema automatically.

Principally this is possible, by using the Schema API but can be difficult to implement it right. Known requirements and challenges are:

1. Solr provides various different field types for different kinds of data. For instance, an integer could be derived from an Int, TrieInt (version < 7.0) or Pint (version >= 7.0) field. This requires a comprehensive fieldtype-datatype mapping.
2. Requires parsing of response json, as the response parsing of Schema API is not really realized in SolrJ
3. Dynamic fields could become a problem

Issue Links

• is related to: NIFI-3248 GetSolr can miss recently updated documents