[HIVE-24524] LLAP ShuffleHandler: upgrade to Netty4 and remove Netty3 dependency from hive where it's possible

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 11/Dec/20 07:57
• Updated: 17/Nov/22 08:48
• Resolved: 21/Apr/21 08:13

Description

Tez already has a WIP patch for upgrading its shuffle handler to netty4. Netty4 is a possible performance improvement compared to Netty3. However, the refactor is not trivial, TEZ-4157 covers that more or less (the code bases are very similar).

Background:
netty4 migration guideline: https://netty.io/wiki/new-and-noteworthy-in-4.0.html
articles of possible performance improvement:
https://blog.twitter.com/engineering/en_us/a/2013/netty-4-at-twitter-reduced-gc-overhead.html
https://developer.squareup.com/blog/upgrading-a-reverse-proxy-from-netty-3-to-4/

some other notes: Netty3 is EOL since 2016:
https://netty.io/news/2016/06/29/3-10-6-Final.html

Issue Links

• is related to: HIVE-25013 Remove Netty3 dependency from druid-handler once Druid upgrades fully to Netty4
• is related to: TEZ-4157 ShuffleHandler: upgrade to Netty4 and remove Netty3 dependency from tez
• is related to: HIVE-18574 LLAP: Ship netty3 as part of LLAP install tarball
• is related to: HADOOP-11219 [Umbrella] Upgrade to netty 4
• links to: GitHub Pull Request #1778