[FLINK-28027] Initialise Async Sink maximum number of in flight messages to low number for rate limiting strategy

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.15.0, 1.15.1
• Fix Version/s: 1.15.2
• Component/s: Connectors / Common, Connectors / Kinesis
• Labels: pull-request-available
• Assignee: Hong Liang Teoh
• Reporter: Zichen Liu
• Created: 13/Jun/22 16:28
• Updated: 11/Aug/22 10:11
• Resolved: 11/Aug/22 10:11

Description

Background

In the AsyncSinkWriter, we implement a rate limiting strategy.

The initial value for the maximum number of in flight messages is set extremely high (maxBatchSize * maxInFlightRequests).

However, in accordance with the AIMD strategy, the TCP implementation for congestion control has found a small value to start with [is better](https://en.wikipedia.org/wiki/TCP_congestion_control#Slow_start).

Suggestion

A better default might be:

• maxBatchSize
• maxBatchSize / parallelism

Issue Links

• is fixed by: FLINK-28487 Introduce configurable RateLimitingStrategy for Async Sink
• links to: GitHub Pull Request #20509
• links to: GitHub Pull Request #20543