Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15515

Error Handling in Running SQL Directly On Files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • SQL
    • None

    Description

      For ORC source format, we are reporting the strange error message when we did not enable Hive support:

      Table or view not found: `org.apache.spark.sql.hive.orc`.`file_path`
      

      The example query is like

            sql(s"select id from `org.apache.spark.sql.hive.orc`.`file_path`")
      

      Instead, we should issue the error message like:

      "The ORC data source must be used with Hive support enabled"
      

      For the Avro format, we still report the error message like:

      Table or view not found: `com.databricks.spark.avro`.`file_path`
      

      The example query is like

            sql(s"select id from `avro`.`file_path`")
            sql(s"select id from `com.databricks.spark.avro`.`file_path`")
      

      The desired message should be like:

      Failed to find data source: avro. Please use Spark package http://spark-packages.org/package/databricks/spark-avro"
      

      Attachments

        Activity

          People

            smilegator Xiao Li
            smilegator Xiao Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: