[HADOOP-15264] AWS "shaded" SDK 1.11.271 is pulling in netty 4.1.17

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0, 3.0.3
• Component/s: fs/s3
• Target Version/s: 3.1.0, 3.0.2
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 26/Feb/18 17:26
• Updated: 26/May/18 14:46
• Resolved: 28/Feb/18 13:11

Description

The latest versions of the AWS Shaded SDK are declaring a dependency on netty 4.1.17

[INFO] +- org.apache.hadoop:hadoop-aws:jar:3.2.0-SNAPSHOT:compile
[INFO] |  \- com.amazonaws:aws-java-sdk-bundle:jar:1.11.271:compile
[INFO] |     +- io.netty:netty-codec-http:jar:4.1.17.Final:compile
[INFO] |     +- io.netty:netty-codec:jar:4.1.17.Final:compile
[INFO] |     +- io.netty:netty-handler:jar:4.1.17.Final:compile
[INFO] |     +- io.netty:netty-buffer:jar:4.1.17.Final:compile
[INFO] |     +- io.netty:netty-common:jar:4.1.17.Final:compile
[INFO] |     +- io.netty:netty-transport:jar:4.1.17.Final:compile
[INFO] |     \- io.netty:netty-resolver:jar:4.1.17.Final:compile

We either exclude these or roll back HADOOP-15040.

Issue Links

• is broken by: HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE in 1.11.199 bug spams logs w/ Yarn Log Aggregation