Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-16828

Support round-robin scheduling with failover for Sinks with distributed collector

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.1
    • 2.5.0
    • amvari-me
    • None

    Description

      • Initial set of collectors is configured in the configuration files
      • Find available collectors by connecting to zookeeper thereafter
      • Remember available collectors, refresh this information only when one collector cannot be reached with a very low frequency of checks, example: random interval between 10-12 minutes, check if a new collector is available. Set a low client side zk timeout.
      • Round robin the write between the collector choosing the first one at random
      • If a write timed out, choose the next available collector, remember the attempts with the first one
      • Set a configurable attempt count for failed connector (default = 3), after which the failed connector is no longer in the available collectors list.
      • The next retry will be triggered after refresh with zookeeper is successful
      • If no failed collectors available, zk refresh interval should be chosen randomly between 1-2 minutes.

      Attachments

        1. AMBARI-16828.patch
          143 kB
          Siddharth Wagle

        Issue Links

          Activity

            People

              swagle Siddharth Wagle
              swagle Siddharth Wagle
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: