Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Using the alternative FileSystem implementations like S3 and Azure requires that the relevant jars are made available on the runtime classpath for Hadoop processes. In a typical distro layout, these jars are in the tools directory, and thus not part of the default classpath. The process of making these jars available on the classpath needs to be documented more clearly.
Attachments
Issue Links
- is related to
-
SPARK-7442 Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has broken S3 filesystem access
- Resolved
- relates to
-
SPARK-7481 Add spark-hadoop-cloud module to pull in object store support
- Resolved