Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2506

LoadBalancingRpcClient should allow for preferred hosts in selector

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Client SDK
    • Labels:
      None

      Description

      The current HostSelector interface relies on an iterator to return the next host without regard to the incoming event data. For clients that use internal caches to speed processing or were otherwise optimized for specific events, it would be nice if host selection could depend on a quick examination by the selector beforehand. Allowing support for host preferences (stickiness) would keep individual caches much tighter but would necessitate replacing the iterator with something that is provided with the actual event:

      public void append(Event event) throws EventDeliveryException {
      
           boolean eventSent = false;
           HostProvider provider = selector.getHostProvider();
      
           HostInfo info = provider.hostFor(event);	// host-specific per event
           int attempts = 0;
      
           while (info != null && attempts < MAX) {
             RpcClient client;
             attempts++;
      	
             try {
               client = getClient(host);
               client.append(event);
               eventSent = true;
               break;
      
             } catch (Exception ex) {
              selector.informFailure(host);
              LOGGER.warn("Failed to send event to host " + host, ex);
             }
      	info = provider.hostFor(event);	// try another (if available)
          }
      

      The new HostProvider interface could wrap the current round-robin and random allocation strategies by default to avoid breaking existing code while others could take advantage of event specificity to chose the host.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              hooperbloob brian remedios

              Dates

              • Created:
                Updated:

                Issue deployment