[SPARK-42041] DataFrameReader should support list of paths

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Connect, PySpark
• Assignee: Ruifeng Zheng
• Reporter: Ruifeng Zheng
• Created: 13/Jan/23 02:14
• Updated: 14/Jan/23 06:45
• Resolved: 14/Jan/23 06:45

Description

for example,

    def load(
        self,
        path: Optional[PathOrPaths] = None,
        format: Optional[str] = None,
        schema: Optional[Union[StructType, str]] = None,
        **options: "OptionalPrimitiveType",
    ) -> "DataFrame":

in PySpark support a list of pathes

Issue Links

• links to: [Github] Pull Request #39553 (zhengruifeng)
• links to: [Github] Pull Request #39553 (zhengruifeng)