[SPARK-6898] Special chars in column names is broken

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: SQL
• Assignee: Wenchen Fan
• Reporter: Wenchen Fan
• Created: 14/Apr/15 08:41
• Updated: 27/Jul/15 16:36
• Resolved: 15/Apr/15 20:39

Description

This function is added a long time ago, but it's not complete, it will fail if we have "." inside column name.

test("SPARK-3483 Special chars in column names") {
    val data = sparkContext.parallelize(
      Seq("""{"key?number1": "value1", "key.number2": "value2"}"""))
    jsonRDD(data).registerTempTable("records")
    sql("SELECT `key?number1`, `key.number2` FROM records")
  }

this test will fail.

Issue Links

• is duplicated by: SPARK-5632 not able to resolve dot('.') in field name
• is duplicated by: SPARK-2775 HiveContext does not support dots in column names.
• relates to: SPARK-9371 Special chars in column names is broken in HiveContext
• links to: [Github] Pull Request #5511 (cloud-fan)