Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2534

Flume tests fail with java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets

    Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: build
    • Labels:
      None
    • Environment:

      Ubuntu 14.04, jdk 1.7

      Description

      I have removed –DskipTests option at bigtop/bigtop-packages/src/common/flume/do-compound-build and tried to run ./gradlew flume-deb.

      TestFlumeAuthenticator test case is failing in flume-ng-auth with below error :

      org.apache.flume.auth.TestFlumeAuthenticator Time elapsed: 7161 sec <<< ERROR!
      java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets
      at org.apache.hadoop.minikdc.MiniKdc.initKDCServer(MiniKdc.java:451)
      at org.apache.hadoop.minikdc.MiniKdc.start(MiniKdc.java:320)
      at org.apache.flume.auth.TestFlumeAuthenticator.startMiniKdc(TestFlumeAuthenticator.java:49)

      It is trying to execute "r = new BufferedReader(new InputStreamReader(is2, Charsets.UTF_8));" at line 451 in hadoop/hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java of hadoop-2.7.2 version.

      It is failing because “org/apache/commons/io/Charsets.java” is available in common-io package > 2.3 version but it is downloading commons-io(2.1) version because of following dependency in flume-1.6.0/pom.xml

      <dependency>
      <groupId>commons-io</groupId>
      <artifactId>commons-io</artifactId>
      <version>2.1</version>
      </dependency>

      There are even other test cases that are failing with “java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets” error.

      Can we change the version of commons-io(2.4) in flume-1.6.0/pom.xml ?

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          Dunno where Flume is now nowadays and if any new releases are available, but ideally I'd prefer to fix this upstream.

          Show
          cos Konstantin Boudnik added a comment - Dunno where Flume is now nowadays and if any new releases are available, but ideally I'd prefer to fix this upstream.
          Hide
          oflebbe Olaf Flebbe added a comment -

          We are now at flume 1.7 in bigtop.

          Anyway, the commons-io version is still at 2.1

          Will try to reproduce.

          Show
          oflebbe Olaf Flebbe added a comment - We are now at flume 1.7 in bigtop. Anyway, the commons-io version is still at 2.1 Will try to reproduce.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Yep, can reproduce. There is an upstream issue open:
          FLUME-2662.

          I will write an email to dev@flume.apache.org

          Show
          oflebbe Olaf Flebbe added a comment - Yep, can reproduce. There is an upstream issue open: FLUME-2662 . I will write an email to dev@flume.apache.org
          Hide
          oflebbe Olaf Flebbe added a comment -

          Committed. Thanks for reporting!

          Show
          oflebbe Olaf Flebbe added a comment - Committed. Thanks for reporting!

            People

            • Assignee:
              oflebbe Olaf Flebbe
              Reporter:
              divyasree_julakanti Divya sree
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development