Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13029

Have FairCallQueue try all lower priority sub queues before backoff

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ipc
    • Labels:

      Description

      Currently if FairCallQueue and backoff are enabled, backoff will kick in as soon as the assigned sub queue is filled up.

        /**
         * Put and offer follow the same pattern:
         * 1. Get the assigned priorityLevel from the call by scheduler
         * 2. Get the nth sub-queue matching this priorityLevel
         * 3. delegate the call to this sub-queue.
         *
         * But differ in how they handle overflow:
         * - Put will move on to the next queue until it lands on the last queue
         * - Offer does not attempt other queues on overflow
         */
      

      Seems it is better to try lower priority sub queues when the assigned sub queue is full, just like the case when backoff is disabled. This will give regular users more opportunities and allow the cluster to be configured with smaller call queue length. Chris Li, [~arpitagarwal], what do you think?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mingma Ming Ma
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: