Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2424

allow index timestamp format option for ES sink

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      I'm using a flume sink to elastic search with a "logstash" index name.

      Flume ES sink creates an index with YYYY-MM-DD format.
      Logstash and kibana logstash uses YYYY.MM.DD format.

      The workaround is to replace the timestamp format in kibana logstash dashboard as described in http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/

      Is it possible to add an option in the conf to choose the timestamp format?
      I find a git repo that allow that but it is not in the flume distrib:
      https://github.com/DevOps-TangoMe/flume-elasticsearch

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                domak Christophe Domas
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: