[SPARK-25999] make-distribution.sh failure with --r and -Phadoop-provided

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 2.3.2, 2.4.0
• Component/s: Build
• Assignee: Unassigned
• Reporter: shanyu zhao
• Created: 09/Nov/18 23:07
• Updated: 05/Sep/19 23:58
• Resolved: 04/Dec/18 14:46

Description

It is not possible to build a distribution that doesn't contain hadoop dependencies but include SparkR. This is because R/check_cran.sh builds R document which depends on hadoop dependencies in assembly/target/scala-xxx/jars folder.

To reproduce:

MAVEN_BUILD_OPTS="-Dmaven.javadoc.skip=true -Pyarn -Phadoop-2.7 -Phive -Psparkr -Phadoop-provided"

./dev/make-distribution.sh --tgz --r $MAVEN_BUILD_OPTS

 

Error:

• creating vignettes ... ERROR
  ...
  Error: A JNI error has occurred, please check your installation and try again
  Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger

Issue Links

• links to: [Github] Pull Request #22997 (shanyu)
• links to: [Github] Pull Request #22997 (shanyu)
• links to: GitHub Pull Request #22997