Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16126

Better Error Message When using DataFrameReader without `path`

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      When users do not specify the path in `DataFrameReader` APIs, it can get a confusing error message. For example,

      spark.read.json()
      

      Error message:

      Unable to infer schema for JSON at . It must be specified manually;
      

      After the fix, the error message will be like:

      'path' is not specified
      

      Another major goal of this PR is to add test cases for the latest changes in https://github.com/apache/spark/pull/13727.

      • orc read APIs
      • illegal format name
      • save API - empty path or illegal path
      • load API - empty path
      • illegal compression
      • fixed a test case in the existing test case `prevent all column partitioning`

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              smilegator Xiao Li
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: