Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
In an instantiated examples archetype, with -P apex-runner, Apex depends on Hadoop 2.6.0 and this is bundles into the examples jar.
In order to get this to run on Hadoop 2.7.3 I added this to the profile:
<properties> <hadoop.version>2.7.3</hadoop.version> </properties> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-yarn-client</artifactId> <version>${hadoop.version}</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>${hadoop.version}</version> </dependency> </dependencies>
It is not clear to me what the best path is, here. Clearly the way we bundle is brittle and probably not the recommended best practice. But also perhaps the deps of the runner can be modified to provided.
Attachments
Issue Links
- duplicates
-
BEAM-3240 Quickstart examples archetype dependencies conflict for Apex on YARN
- Resolved