Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2877

Add ability to configure avro sinks with a LoadBalancing RPC client

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      In addition to load balancing sink groups, have the ability to utilize LoadBalancing RPC client as a sink. This minimizes configuration from n^2 to n sink specifications to enable load balancing.

      A sample of what the configuration would look like would be the below.

      <agentName>.sinks.<sinkName>.channel = ch1
      <agentName>.sinks.<sinkName>.type = avro
      <agentName>.sinks.<sinkName>.client.type = default_loadbalance
      <agentName>.sinks.<sinkName>.hosts = h1 h2 h3
      <agentName>.sinks.<sinkName>.hosts.h1 = host1.domain.com:19211
      <agentName>.sinks.<sinkName>.hosts.h2 = host2.domain.com:19211
      <agentName>.sinks.<sinkName>.hosts.h3 = host3.domain.com:19211
      <agentName>.sinks.<sinkName>.host-selector = random
      <agentName>.sinks.<sinkName>.backoff = true
      <agentName>.sinks.<sinkName>.batch-size = 100
      <agentName>.sinks.<sinkName>.connect-timeout=30000
      <agentName>.sinks.<sinkName>.request-timeout=30000

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jhambleton Jordan Hambleton
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: