[SPARK-17585] PySpark SparkContext.addFile supports adding files recursively

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: PySpark, Spark Core
• Assignee: Yanbo Liang
• Reporter: Yanbo Liang
• Created: 18/Sep/16 14:23
• Updated: 09/Oct/16 08:42
• Resolved: 21/Sep/16 08:38

Description

Users would like to add a directory as dependency in some cases, they can use SparkContext.addFile with argument recursive=true to recursively add all files under the directory by using Scala. But Python users can only add file not directory, we should also make it supported.

Issue Links

• is duplicated by: SPARK-17287 PySpark sc.AddFile method does not support the recursive keyword argument
• links to: [Github] Pull Request #15140 (yanboliang)