[FLUME-2206] ElasticSearchSink ttl field modification to mimic Elasticsearch way of specifying TTL

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0
• Component/s: Sinks+Sources
• Labels: features patch
• Assignee: Dib Ghosh
• Reporter: Dib Ghosh
• Created: 08/Oct/13 18:30
• Updated: 31/Oct/13 06:14
• Resolved: 31/Oct/13 06:14

Description

According to current ElasticSearch documentation of TTL:

"In case you do not specify a time unit like d (days), m (minutes), h (hours), ms (milliseconds) or w (weeks), milliseconds is used as default unit.
If no default is set and no _ttl value is given then the document has an infinite _ttl and will not expire."

Current flume ElasticSearchSink index TTL configuration:

agent.sinks.myESsink.ttl = <some integer value> and defaults to 5 days.

It will be good to have a qualifier like "d" / "s" / "m" / "w" / "h" to mimic the TTL configuration in ElasticSearch mapping.

When user specifies agent.sinks.myESsink.ttl = "3w" index TTL should be set to 3 weeks while agent.sinks.myESsink.ttl = 3 will set the TTL to 3 days.