[NIFI-1576] Incorporate feedback/improvements to Elasticsearch processors

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Abandoned
• Assignee: Unassigned
• Reporter: Matt Burgess
• Created: 29/Feb/16 16:29
• Updated: 10/Nov/22 22:53
• Resolved: 10/Nov/22 22:53

Description

Since the release of Apache NiFi 0.5.0 containing FetchElasticsearch and PutElasticsearch processors, there have been some improvements identified that would add to the usability, flexibility, and power of the Elasticsearch processor(s). This Jira should capture those improvements for incorporation in a future release.

Three initial observations:
1) FetchElasticsearch currently requires input, but it is possible to configure the processor such that no information is required from an incoming flow file. The processor should be changed to allow (but not require) input.

2) FetchElasticsearch specifies a "Document Identifier" property which may or may not contain Expression Language to determine the document ID for fetching. However PutElasticsearch only works when the IDs are unique, so it specifies a "Identifier Attribute" which forces the user to point at an existing attribute. The documentation should be very clear about what the properties are and why/how they're used.

3) Related to #2, PutElasticsearch's "Identifier Attribute" is required and a value for that attribute is currently required. Instead, if no value is present, the processor should leverage the auto-generated ID feature of the Elasticsearch API.

Issue Links

• is related to: NIFI-1666 PutElasticsearch doesn't evaluate Expression Language correctly for Index and Document Type
• is superceded by: NIFI-9388 Elasticsearch processor to fetch a single document by _id using the ElasticsearchClientService
• relates to: NIFI-1594 Add option to bulk using Index or Update to PutElasticsearch