[FLINK-7258] IllegalArgumentException in Netty bootstrap with large memory state segment size

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.3.1
• Fix Version/s: 1.3.2, 1.4.0
• Component/s: Network
• Assignee: Ufuk Celebi
• Reporter: Ufuk Celebi
• Created: 24/Jul/17 15:32
• Updated: 27/Sep/17 11:43
• Resolved: 25/Jul/17 14:11

Description

In NettyBootstrap we configure the low and high watermarks in the following order:

bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, config.getMemorySegmentSize() + 1);
bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 2 * config.getMemorySegmentSize());

When the memory segment size is higher than the default high water mark, this throws an `IllegalArgumentException` when a client tries to connect. Hence, this unfortunately only happens during runtime when a intermediate result is requested. This doesn't fail the job, but logs a warning and ignores the failed configuration attempt, potentially resulting in degraded performance because of a lower than expected watermark.

A simple fix is to first configure the high water mark and only then configure the low watermark.

Issue Links

• breaks: FLINK-7701 IllegalArgumentException in Netty bootstrap with small memory state segment size
• links to: GitHub Pull Request #4391