Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2207

HDFS file suffix style index suffix in ElasticSearchSink

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Currently ElasticSearchSink indexName is appended with "YYYY-MM-DD" to generate actual ElasticSearch index name and allow index rolling on a daily basis. This works well for simpler use cases but is restrictive imho. User should be able to add indexSuffix as per their own use case and also keep the indexSuffix empty if requirement demands.

      agent.sinks.essink.indexName = flume becomes flume-2013-10-10.

      It will be really useful to have HDFS Sink filepath style indexSuffix naming i.e. "YYYY" will roll the indexes yearly, "YYYY-MM" will create indices monthly.

        Attachments

        1. FLUME-2207-1.patch
          0.8 kB
          Satoshi Iijima

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dib.ghosh Dib Ghosh
              • Votes:
                6 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: