Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9170

ORC data source creates a schema with lowercase table names

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.1
    • 1.6.0
    • SQL
    • None

    Description

      Steps to reproduce:

      sqlContext.range(0, 10).select('id as "Acol").write.format("orc").save("/tmp/foo")
      
      sqlContext.read.format("orc").load("/tmp/foo").schema("Acol")
      //java.lang.IllegalArgumentException: Field "Acol" does not exist.
      
      sqlContext.read.format("orc").load("/tmp/foo").schema("acol")
      //org.apache.spark.sql.types.StructField = StructField(acol,LongType,true)
      
      sqlContext.read.format("orc").load("/tmp/foo").select("Acol").show()
      //+----+
      |Acol|
      +----+
      |   1|
      |   5|
      |   3|
      |   4|
      |   7|
      |   2|
      |   6|
      |   8|
      |   9|
      |   0|
      +----+
      

      Attachments

        Activity

          People

            viirya L. C. Hsieh
            prudenko Peter Rudenko
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: