[SPARK-30619] org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 2.4.2, 2.4.4
• Component/s: Build

Environment

Spark on kubernetes

• Assignee: Unassigned
• Reporter: Abhishek Rao
• Created: 23/Jan/20 10:20
• Updated: 12/Dec/22 18:10
• Resolved: 10/Feb/20 02:49

Description

We're using spark-2.4.4-bin-without-hadoop.tgz and executing Java Word count (org.apache.spark.examples.JavaWordCount) example on local files.

But we're seeing that it is expecting org.slf4j.Logger and org.apache.commons.collections classes to be available for executing this.

We expected the binary to work as it is for local files. Is there anything which we're missing?

Issue Links

• is duplicated by: SPARK-30723 Executing the example on https://spark.apache.org/docs/latest/running-on-yarn.html fails