[SPARK-31763] DataFrame.inputFiles() not Available

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.5
• Fix Version/s: 3.1.0
• Component/s: PySpark
• Assignee: Rakesh Raushan
• Reporter: Felix Kizhakkel Jose
• Created: 19/May/20 20:29
• Updated: 12/Dec/22 18:11
• Resolved: 28/May/20 00:52

Description

I have been trying to list inputFiles that compose my DataSet by using PySpark 

spark_session.read
.format(sourceFileFormat)
.load(S3A_FILESYSTEM_PREFIX + bucket + File.separator + sourceFolderPrefix)
.inputFiles();

but I get an exception saying inputFiles attribute not present. But I was able to get this functionality with Spark Java. 

So is this something missing in PySpark?

Issue Links

• links to: [Github] Pull Request #28652 (iRakson)