[FLUME-2884] Tag not allowed in ElasticSearch indexNameBuilder properties

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Julien Sobczak
• Created: 18/Feb/16 16:39
• Updated: 18/Feb/16 16:46

Description

Note: The patch FLUME - 2207 is required to reproduce the issue.

The following code causes the flume agent to crash a stack trace when starting (illegal pattern passed to FastDateFormat) :

agent1.sinks.sinkEls.indexNameBuilder.dateFormat = %{format}

In practice, we need a variable to customize the index rotation (daily, weekly, monthly) from a header. (the header is defined in a morphline in our case, configured as an interceptor on the source)

The problem happens because the FastDateFormat is instanciated in the method configure. We could delay its instanciation inside the method getIndexName without decrease the performance of the agent (FastDateFormat uses caching extensively). I added a patch to present this implementation.

Issue Links

• depends upon: FLUME-2207 HDFS file suffix style index suffix in ElasticSearchSink