[FLUME-2015] ElasticSearchSink: need access to IndexRequestBuilder instance during flume event processing

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.3.1
• Fix Version/s: 1.4.0
• Component/s: Sinks+Sources
• Assignee: Tim Bacon
• Reporter: Tim Bacon
• Created: 22/Apr/13 10:00
• Updated: 28/Oct/13 21:21
• Resolved: 10/May/13 20:29

Description

I need more control over the indexing performed by the ElasticSearchSink – in particular I need access to an IndexRequestBuilder instance during flume event processing. The interactions between ElasticSearchSink and ElasticSearchEventSerializer currently do not make this possible.

I have authored a patch that meets my needs and maintains backwards-compatibility for existing users of the sink. It is available at https://github.com/prime8/flume/commit/1056c129b10c95cee50e0c3f77e309668f82bfc6 – please let me know if it can be pulled into the main ASF source tree!

Thanks

Issue Links

• is related to: FLUME-1782 Elastic Search sink does not use UTC to determine the correct index to write to.
• is related to: FLUME-1992 ElasticSearch dependency is marked optional
• relates to: FLUME-1972 ElasticSearchSink now allows the id of the document to be provided by a custom class
• links to: Code Review Board
• links to: Documentation Review Board