[NIFI-7801] Add acknowledgement check to Splunk

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: Extensions
• Assignee: Simon Bence
• Reporter: Simon Bence
• Created: 11/Sep/20 12:04
• Updated: 02/Mar/21 17:03
• Resolved: 26/Jan/21 20:25

Description

Currently there is no proper way to utilise HTTP Event Collector Indexer Acknowledgment functionality when sending data into Splunk. It would be beneficial to extend current functionality with the client side handling of this capability.

There are multiple possible approaches, like:

• Extend PutSplunk with HTTP protocol (downside: the current behaviour does not really fits with adding HTTP support)
• Extend InvokeHTTP with (downside: this is not a Splunk specific processor)
• Adding a new processor, PutSplunkHTTP

Adding a new, specific processor with limited possibilities for customise HTTP requests and the capability to "follow up" on acknowledgements looks like viable approach. The highlights are:

• The processor should send incoming flowfile content to Splunk, in similar fashion as PutSplunk behaves
• This should happen via HTTP protocol
• The processor must support channel and channel id
• The processor should store incoming ackId-s
• There should be a mechanism polling ackId-s and based on the Splunk's answer (or possible timeout) it should manage this collection of ids. Based on the outcome, the processor should emit on the an appropriate relationship.

From Splunk side, this is the initial expectation: https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/AboutHECIDXAck

Issue Links

• relates to: NIFI-8176 Add indicator flag for Splunk acknowledgements
• links to: GitHub Pull Request #4714