Description
The following issue was reported:
016-02-17 15:23:11,814 ERROR [Timer-Driven Process Thread-7] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=0575cb57-786c-45d9-b9ef-19c15cc01290] Failed to writ
e to HDFS due to java.lang.NoClassDefFoundError: org/apache/http/Consts
2016-02-17 15:23:11,816 ERROR [Timer-Driven Process Thread-7] o.apache.nifi.processors.hadoop.PutHDFS
java.lang.NoClassDefFoundError: org/apache/http/Consts
at org.apache.http.client.utils.URIBuilder.digestURI(URIBuilder.java:181) ~[httpclient-4.4.1.jar:4.4.1]
at org.apache.http.client.utils.URIBuilder.<init>(URIBuilder.java:82) ~[httpclient-4.4.1.jar:4.4.1]
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createURL(KMSClientProvider.java:368) ~[hadoop-common-2.6.2.jar:na]
From looking into this, we have the following transitive dependencies from the hadoop libraries nar:
- net.java.dev.jets3t:jets3t:jar:0.9.0:compile
[INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.4.1:compile
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.1.2:compile
I believe the 4.4.1 of httpclient is coming from the top level pom which has a dependency management block that forces the client to 4.4.1 and doesn't for core to 4.4.1.
If we look at httpcore 4.x branch it has the missing Consts class:
https://github.com/apache/httpcore/tree/4.4.x/httpcore/src/main/java/org/apache/http
and the 4.1.x branch does not:
https://github.com/apache/httpcore/tree/4.1.x/httpcore/src/main/java/org/apache/http
We either need to force both to 4.4.1, or force only the hadoop libraries nar to both be 4.1.2 if that is what it normally brings in.
Attachments
Issue Links
- relates to
-
NIFI-2828 SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport
-
- Resolved
-