[SPARK-32815] Fix LibSVM data source loading error on file paths with glob metacharacters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.6, 3.0.1, 3.1.0
• Fix Version/s: 2.4.8, 3.0.2, 3.1.0
• Component/s: MLlib
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 08/Sep/20 06:28
• Updated: 09/Sep/20 21:01
• Resolved: 08/Sep/20 14:15

Description

SPARK-32810 fixed a long standing bug in a few Spark built-in data sources that fails to read files whose names contain glob metacharacters, such as [, ], {, }, etc.

CSV and JSON data source on the Spark side were affected. We've also noticed that the LibSVM data source had the same code pattern that leads to the bug, so the fix https://github.com/apache/spark/pull/29659 included a fix for that data source as well, but it did not include a test for the LibSVM data source.

This ticket tracks adding a test case for LibSVM, similar to the ones for CSV/JSON, to verify whether or not the fix works as intended.

Issue Links

• relates to: SPARK-32810 CSV/JSON data sources should avoid globbing paths when inferring schema
• links to: [Github] Pull Request #29670 (MaxGekk)
• links to: [Github] Pull Request #29675 (MaxGekk)
• links to: [Github] Pull Request #29675 (MaxGekk)
• links to: [Github] Pull Request #29676 (MaxGekk)
• links to: [Github] Pull Request #29676 (MaxGekk)
• links to: [Github] Pull Request #29678 (MaxGekk)