[BIGTOP-3714] Fix error of DataNode httpserver due to conflicting dependency on Netty in branch-3.1

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.1
• Component/s: hadoop

Environment

Debian 11
temurin-8-jdk
bigtop-groovy 2.5.4-1
bigtop-jsvc 1.2.4-1
bigtop-utils 3.1.0-1
hadoop 3.2.3-1
hadoop-client 3.2.3-1
hadoop-hdfs 3.2.3-1
hadoop-hdfs-datanode 3.2.3-1
hadoop-mapreduce 3.2.3-1
hadoop-yarn 3.2.3-1
hadoop-yarn-nodemanager 3.2.3-1

• Assignee: Masatake Iwasaki
• Reporter: Zoltan Lanyi
• Created: 01/Jul/22 08:14
• Updated: 05/Jan/23 12:21
• Resolved: 04/Jul/22 03:35

Description

Hi,
if i make a request to datanode http server address with curl then i get error:

2022-07-01 10:00:39,549 WARN io.netty.channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.NoClassDefFoundError: Could not initialize class io.netty.handler.codec.http.HttpVersion

Netty Jar Files

find /usr/lib/ -name netty*
/usr/lib/hadoop/lib/netty-3.10.6.Final.jar
/usr/lib/hadoop/lib/netty-transport-native-epoll-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-buffer-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-codec-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-common-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-handler-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-resolver-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-transport-native-unix-common-4.1.50.Final.jar
/usr/lib/hadoop/lib/netty-transport-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-all-4.1.68.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-3.10.6.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-common-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-transport-native-unix-common-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-transport-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-codec-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-handler-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-buffer-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-resolver-4.1.50.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-transport-native-epoll-4.1.50.Final.jar
/usr/lib/spark/jars/netty-all-4.1.51.Final.jar
/usr/lib/zookeeper/lib/netty-buffer-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-handler-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-transport-native-epoll-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-transport-native-unix-common-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-codec-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-common-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-resolver-4.1.50.Final.jar
/usr/lib/zookeeper/lib/netty-transport-4.1.50.Final.jar

Maybe is not relevant, but on Release 3.0.0 very less netty are available (and there is working the status page):

find /usr/lib/ -name netty*
/usr/lib/hadoop/lib/netty-3.10.6.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-3.10.6.Final.jar
/usr/lib/hadoop-hdfs/lib/netty-all-4.1.48.Final.jar
/usr/lib/spark/jars/netty-all-4.1.47.Final.jar
/usr/lib/zookeeper/lib/netty-3.10.6.Final.jar

Thanks in advance,
Zoltan

Issue Links

• is duplicated by: BIGTOP-3743 Timeout for upload files though webhdfs
• relates to: BIGTOP-3715 Add TestWebHDFS to smoke-tests
• links to: GitHub pull request #927