[FLUME-2424] allow index timestamp format option for ES sink

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.5.0
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Christophe Domas
• Created: 16/Jul/14 15:23
• Updated: 08/Oct/22 17:39
• Resolved: 08/Oct/22 17:39

Description

I'm using a flume sink to elastic search with a "logstash" index name.

Flume ES sink creates an index with YYYY-MM-DD format.
Logstash and kibana logstash uses YYYY.MM.DD format.

The workaround is to replace the timestamp format in kibana logstash dashboard as described in http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/

Is it possible to add an option in the conf to choose the timestamp format?
I find a git repo that allow that but it is not in the flume distrib:
https://github.com/DevOps-TangoMe/flume-elasticsearch

Issue Links

• relates to: FLUME-2391 Turn Elasticsearch sink index timestamp parametrizable
• relates to: FLUME-2207 HDFS file suffix style index suffix in ElasticSearchSink