Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32815

Fix LibSVM data source loading error on file paths with glob metacharacters

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.6, 3.0.1, 3.1.0
    • Fix Version/s: 2.4.8, 3.0.2, 3.1.0
    • Component/s: MLlib
    • Labels:
      None

      Description

      SPARK-32810 fixed a long standing bug in a few Spark built-in data sources that fails to read files whose names contain glob metacharacters, such as [, ], {, }, etc.

      CSV and JSON data source on the Spark side were affected. We've also noticed that the LibSVM data source had the same code pattern that leads to the bug, so the fix https://github.com/apache/spark/pull/29659 included a fix for that data source as well, but it did not include a test for the LibSVM data source.

      This ticket tracks adding a test case for LibSVM, similar to the ones for CSV/JSON, to verify whether or not the fix works as intended.

        Attachments

          Activity

            People

            • Assignee:
              maxgekk Max Gekk
              Reporter:
              maxgekk Max Gekk
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: