Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9640 RPC Congestion Control with FairCallQueue
  3. HADOOP-10302

Allow CallQueue impls to be swapped at runtime (part 1: internals) Depends on: subtask1

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We wish to swap the active call queue during runtime in order to do performance tuning without restarting the namenode.

      This patch adds only the internals necessary to swap. Part 2 will add a user interface so that it can be used.

        Activity

        Hide
        chrilisf Chris Li added a comment -

        Depends on subtask1

        Swapping is done through the reconfigure(CallQueue<E> newQ) method on custom CallQueue impls. The old call queue is given the responsibility of swapping to the new queue, which it will either accept or reject.

        Both old and new queues are frozen during a swap, not accepting new takes or puts.

        When queues have been swapped (or not in case of failure), they are unfrozen. Producers and consumers waiting on the old queue are awoken and begin drawing from the new queue.

        CallQueueBase is modified in order to handle waiting() while frozen and drawing from the next queue after a successful swap.

        Show
        chrilisf Chris Li added a comment - Depends on subtask1 Swapping is done through the reconfigure(CallQueue<E> newQ) method on custom CallQueue impls. The old call queue is given the responsibility of swapping to the new queue, which it will either accept or reject. Both old and new queues are frozen during a swap, not accepting new takes or puts. When queues have been swapped (or not in case of failure), they are unfrozen. Producers and consumers waiting on the old queue are awoken and begin drawing from the new queue. CallQueueBase is modified in order to handle waiting() while frozen and drawing from the next queue after a successful swap.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12625691/subtask2_runtime_swap_internal.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3490//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12625691/subtask2_runtime_swap_internal.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3490//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12628661/HADOOP-10302.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3567//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12628661/HADOOP-10302.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3567//console This message is automatically generated.
        Hide
        chrilisf Chris Li added a comment -

        swappability is now included in the first patch, so this is not needed.

        Show
        chrilisf Chris Li added a comment - swappability is now included in the first patch, so this is not needed.

          People

          • Assignee:
            chrilisf Chris Li
            Reporter:
            chrilisf Chris Li
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development