Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2576

Elasticsearch sink: customizing dateFormat for IndexNameBuilder does not work

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.5.2
    • Fix Version/s: notrack
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      all

      Description

      I use flume with Elasticsearch sink to insert events in Elasticsearch, and i would like to use this pattern for index name : prefix-YYYY.MM.DD
      I tried to customize the index name in my config file, like this :
      agent_es.sinks.es_dest.indexNameBuilder.dateFormat = yyyy.MM.dd

      But it does not change index name used.
      As I tryed to debug flume code, I found this in class org.apache.flume.sink.elasticsearch.ElasticSearchSink, at line 316:

          Context indexnameBuilderContext = new Context();
          serializerContext.putAll(
                  context.getSubProperties(INDEX_NAME_BUILDER_PREFIX));
      

      It seems that wrong context is used, and following code would work better :

          Context indexnameBuilderContext = new Context();
          indexnameBuilderContext.putAll(
                  context.getSubProperties(INDEX_NAME_BUILDER_PREFIX));
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                frdrolland Frédéric Rolland
              • Votes:
                2 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 10m
                  10m
                  Remaining:
                  Remaining Estimate - 10m
                  10m
                  Logged:
                  Time Spent - Not Specified
                  Not Specified