Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This change introduces a new configuration key used by RPC server to decide whether to send backoff signal to RPC Client when RPC call queue is full. When the feature is enabled, RPC server will no longer block on the processing of RPC requests when RPC call queue is full. It helps to improve quality of service when the service is under heavy load. The configuration key is in the format of "ipc.#port#.backoff.enable" where #port# is the port number that RPC server listens on. For example, if you want to enable the feature for the RPC server that listens on 8020, set ipc.8020.backoff.enable to true.
      Show
      This change introduces a new configuration key used by RPC server to decide whether to send backoff signal to RPC Client when RPC call queue is full. When the feature is enabled, RPC server will no longer block on the processing of RPC requests when RPC call queue is full. It helps to improve quality of service when the service is under heavy load. The configuration key is in the format of "ipc.#port#.backoff.enable" where #port# is the port number that RPC server listens on. For example, if you want to enable the feature for the RPC server that listens on 8020, set ipc.8020.backoff.enable to true.

      Description

      Currently if an application hits NN too hard, RPC requests be in blocking state, assuming OS connection doesn't run out. Alternatively RPC or NN can throw some well defined exception back to the client based on certain policies when it is under heavy load; client will understand such exception and do exponential back off, as another implementation of RetryInvocationHandler.

        Attachments

        1. HADOOP-10597.patch
          19 kB
          Ming Ma
        2. HADOOP-10597-2.patch
          34 kB
          Ming Ma
        3. HADOOP-10597-3.patch
          32 kB
          Ming Ma
        4. HADOOP-10597-4.patch
          34 kB
          Ming Ma
        5. HADOOP-10597-5.patch
          11 kB
          Ming Ma
        6. HADOOP-10597-6.patch
          11 kB
          Ming Ma
        7. HADOOP-10597-branch-2.7.patch
          13 kB
          Zhe Zhang
        8. MoreRPCClientBackoffEvaluation.pdf
          55 kB
          Ming Ma
        9. RPCClientBackoffDesignAndEvaluation.pdf
          169 kB
          Ming Ma

          Activity

            People

            • Assignee:
              mingma Ming Ma
              Reporter:
              mingma Ming Ma
            • Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: