[FLINK-13770] Bump Netty to 4.1.39.Final

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: shaded-8.0, 1.10.0
• Component/s: BuildSystem / Shaded, Runtime / Network
• Labels: pull-request-available
• Assignee: Nico Kruber
• Reporter: Nico Kruber
• Created: 19/Aug/19 10:19
• Updated: 31/Aug/19 11:25
• Resolved: 31/Aug/19 11:25

Description

I quickly went through all the changelogs for Netty 4.1.32 (which we
currently use) to the latest Netty 4.1.39.Final. Below, you will find a
list of bug fixes and performance improvements that may affect us. Nice
changes we could benefit from, also for the Java > 8 efforts. The most
important ones fixing leaks etc are #8921, #9167, #9274, #9394, and the
various CompositeByteBuf fixes. The rest are mostly performance
improvements.

Since we are still early in the dev cycle for Flink 1.10, it would be
nice to update now and verify that the new version works correctly.

Netty 4.1.33.Final
- Fix ClassCastException and native crash when using kqueue transport
(#8665)
- Provide a way to cache the internal nioBuffer of the PooledByteBuffer
to reduce GC (#8603)

Netty 4.1.34.Final
- Do not use GetPrimitiveArrayCritical(...) due multiple not-fixed bugs
related to GCLocker (#8921)
- Correctly monkey-patch id also in whe os / arch is used within library
name (#8913)
- Further reduce ensureAccessible() overhead (#8895)
- Support using an Executor to offload blocking / long-running tasks
when processing TLS / SSL via the SslHandler (#8847)
- Minimize memory footprint for AbstractChannelHandlerContext for
handlers that execute in the EventExecutor (#8786)
- Fix three bugs in CompositeByteBuf (#8773)

Netty 4.1.35.Final
- Fix possible ByteBuf leak when CompositeByteBuf is resized (#8946)
- Correctly produce ssl alert when certificate validation fails on the
client-side when using native SSL implementation (#8949)

Netty 4.1.37.Final
- Don't filter out TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (#9274)
- Try to mark child channel writable again once the parent channel
becomes writable (#9254)
- Properly debounce wakeups (#9191)
- Don't read from timerfd and eventfd on each EventLoop tick (#9192)
- Correctly detect that KeyManagerFactory is not supported when using
OpenSSL 1.1.0+ (#9170)
- Fix possible unsafe sharing of internal NIO buffer in CompositeByteBuf
(#9169)
- KQueueEventLoop won't unregister active channels reusing a file
descriptor (#9149)
- Prefer direct io buffers if direct buffers pooled (#9167)

Netty 4.1.38.Final
- Prevent ByteToMessageDecoder from overreading when !isAutoRead (#9252)
- Correctly take length of ByteBufInputStream into account for
readLine() / readByte() (#9310)
- availableSharedCapacity will be slowly exhausted (#9394)

Issue Links

• blocks: FLINK-13771 Support kqueue Netty transports (MacOS)
• links to: GitHub Pull Request #70