Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25918

LOAD DATA LOCAL INPATH should handle a relative path

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.1, 3.0.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Unfortunately, it seems that we missed this in 2.4.0. In Spark 2.4, if the default file system is not the local file system, LOAD DATA LOCAL INPATH only works in case of absolute paths. This PR aims to fix it to support relative paths. This is a regression in 2.4.0.

      $ ls kv1.txt
      kv1.txt
      
      scala> spark.sql("LOAD DATA LOCAL INPATH 'kv1.txt' INTO TABLE t")
      org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: kv1.txt;
      

        Attachments

          Activity

            People

            • Assignee:
              dongjoon Dongjoon Hyun
              Reporter:
              dongjoon Dongjoon Hyun
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: