[FLUME-2877] Add ability to configure avro sinks with a LoadBalancing RPC client

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.5.0
• Component/s: Configuration, Sinks+Sources
• Assignee: Unassigned
• Reporter: Jordan Hambleton
• Created: 03/Feb/16 15:52
• Updated: 27/Oct/16 21:55

Description

In addition to load balancing sink groups, have the ability to utilize LoadBalancing RPC client as a sink. This minimizes configuration from n^2 to n sink specifications to enable load balancing.

A sample of what the configuration would look like would be the below.

<agentName>.sinks.<sinkName>.channel = ch1
<agentName>.sinks.<sinkName>.type = avro
<agentName>.sinks.<sinkName>.client.type = default_loadbalance
<agentName>.sinks.<sinkName>.hosts = h1 h2 h3
<agentName>.sinks.<sinkName>.hosts.h1 = host1.domain.com:19211
<agentName>.sinks.<sinkName>.hosts.h2 = host2.domain.com:19211
<agentName>.sinks.<sinkName>.hosts.h3 = host3.domain.com:19211
<agentName>.sinks.<sinkName>.host-selector = random
<agentName>.sinks.<sinkName>.backoff = true
<agentName>.sinks.<sinkName>.batch-size = 100
<agentName>.sinks.<sinkName>.connect-timeout=30000
<agentName>.sinks.<sinkName>.request-timeout=30000