Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20574

Throttle number of remote invocation requests on startup or restores with large backlogs

    XMLWordPrintableJSON

Details

    Description

      On startup or restores, the RequestReplyFunction may heavily load the remote functions with multiple concurrent invocation requests if there is a large backlog of restored or historical events to process through.

      The new protocol introduced by FLINK-20265 emphasizes this much more due to the nature of extra invocation roundtrips if the function has state declarations (i.e., the first hoard of concurrent invocations would all fail with an IncompleteInvocationContext and requires invocation patching + state registrations).

      We should think about how to apply throttling to mitigate these scenarios.

      Attachments

        Activity

          People

            tzulitai Tzu-Li (Gordon) Tai
            tzulitai Tzu-Li (Gordon) Tai
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: