Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9716

Add jitter to the exponential backoff in status reporting

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 4.0
    • Fix Version/s: Impala 4.0
    • Component/s: Distributed Exec
    • Labels:
      None
    • Epic Color:
      ghx-label-10

      Description

      When status reports fail, we use exponential backoff when retrying sending them. However, currently the backoff is deterministic, leading to a thundering herd problem where all of the backends for a particular query may try to report at the same time, the coordinator is overwhelmed and rejects some of the rpcs, then the backends all backoff by the same amount and retry sending at the same time, leading the coordinator to be overwhelmed again.

      We can help solve this by adding some random jitter to the exponential backoff time.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                twmarshall Thomas Tauber-Marshall
                Reporter:
                twmarshall Thomas Tauber-Marshall
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: