[SPARK-44081] Simplify PartitionedFileUtil API a little

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: Spark Core
• Assignee: Ryan Johnson
• Reporter: Ryan Johnson
• Created: 16/Jun/23 17:42
• Updated: 16/Jun/23 20:29
• Resolved: 16/Jun/23 20:29

Description

Utility methods in PartitionedFileUtil take both file: FileStatusWithMetadata and filePath: Path, even tho the latter can be obtained easily from the former.

This was originally done for performance reasons, to allow callers to pass a memoized Path and thus avoid the cost of an additional FileStatus.getPath call.

Now that we have FileStatusWithMetadata, we no longer need the redundancy because the new class can capture the path as a lazy val.