[BIGTOP-2534] Flume tests fail with java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Component/s: build

Environment

Ubuntu 14.04, jdk 1.7

• Assignee: Olaf Flebbe
• Reporter: Divya sree
• Created: 14/Sep/16 12:25
• Updated: 29/Mar/17 18:46
• Resolved: 20/Nov/16 19:58

Description

I have removed –DskipTests option at bigtop/bigtop-packages/src/common/flume/do-compound-build and tried to run ./gradlew flume-deb.

TestFlumeAuthenticator test case is failing in flume-ng-auth with below error :

org.apache.flume.auth.TestFlumeAuthenticator Time elapsed: 7161 sec <<< ERROR!
java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets
at org.apache.hadoop.minikdc.MiniKdc.initKDCServer(MiniKdc.java:451)
at org.apache.hadoop.minikdc.MiniKdc.start(MiniKdc.java:320)
at org.apache.flume.auth.TestFlumeAuthenticator.startMiniKdc(TestFlumeAuthenticator.java:49)

It is trying to execute "r = new BufferedReader(new InputStreamReader(is2, Charsets.UTF_8));" at line 451 in hadoop/hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java of hadoop-2.7.2 version.

It is failing because “org/apache/commons/io/Charsets.java” is available in common-io package > 2.3 version but it is downloading commons-io(2.1) version because of following dependency in flume-1.6.0/pom.xml

<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.1</version>
</dependency>

There are even other test cases that are failing with “java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets” error.

Can we change the version of commons-io(2.4) in flume-1.6.0/pom.xml ?

Issue Links

• relates to: BIGTOP-2598 Remove the makeshift patch introduced in BIGTOP-2534
• relates to: FLUME-2662 Upgrade to Commons-IO 2.4