[FLUME-2576] Elasticsearch sink: customizing dateFormat for IndexNameBuilder does not work

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.5.2
• Fix Version/s: notrack
• Component/s: Sinks+Sources

Environment

all

• Assignee: Unassigned
• Reporter: Frédéric Rolland
• Created: 17/Dec/14 17:29
• Updated: 03/Feb/16 03:39

Description

I use flume with Elasticsearch sink to insert events in Elasticsearch, and i would like to use this pattern for index name : prefix-YYYY.MM.DD
I tried to customize the index name in my config file, like this :
agent_es.sinks.es_dest.indexNameBuilder.dateFormat = yyyy.MM.dd

But it does not change index name used.
As I tryed to debug flume code, I found this in class org.apache.flume.sink.elasticsearch.ElasticSearchSink, at line 316:

    Context indexnameBuilderContext = new Context();
    serializerContext.putAll(
            context.getSubProperties(INDEX_NAME_BUILDER_PREFIX));

It seems that wrong context is used, and following code would work better :

    Context indexnameBuilderContext = new Context();
    indexnameBuilderContext.putAll(
            context.getSubProperties(INDEX_NAME_BUILDER_PREFIX));

Issue Links

• duplicates: FLUME-2207 HDFS file suffix style index suffix in ElasticSearchSink