Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2207

HDFS file suffix style index suffix in ElasticSearchSink

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 1.5.0
    • None
    • Sinks+Sources
    • None

    Description

      Currently ElasticSearchSink indexName is appended with "YYYY-MM-DD" to generate actual ElasticSearch index name and allow index rolling on a daily basis. This works well for simpler use cases but is restrictive imho. User should be able to add indexSuffix as per their own use case and also keep the indexSuffix empty if requirement demands.

      agent.sinks.essink.indexName = flume becomes flume-2013-10-10.

      It will be really useful to have HDFS Sink filepath style indexSuffix naming i.e. "YYYY" will roll the indexes yearly, "YYYY-MM" will create indices monthly.

      Attachments

        1. FLUME-2207-1.patch
          0.8 kB
          Satoshi Iijima

        Issue Links

          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. FLUME-2884 Tag not allowed in ElasticSearch indexNameBuilder properties

          • Major - Major loss of function.
          • Patch Available
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. FLUME-2576 Elasticsearch sink: customizing dateFormat for IndexNameBuilder does not work

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. FLUME-2876 ElasticSearchSink: indexnameBuilderContext.putAll bug fixes

          • Major - Major loss of function.
          • Patch Available

          Improvement - An improvement or enhancement to an existing feature or task. FLUME-2391 Turn Elasticsearch sink index timestamp parametrizable

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is part of

          Bug - A problem which impairs or prevents the functions of the product. FLUME-2435 elasticsearchsink: IndexNameBuilder configuration fails

          • Major - Major loss of function.
          • Patch Available
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. FLUME-2424 allow index timestamp format option for ES sink

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Review Board

          Activity

            People

              Unassigned Unassigned
              dib.ghosh Dib Ghosh
              Votes:
              6 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: