[SPARK-2775] HiveContext does not support dots in column names.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: SQL
• Target Version/s: 1.4.0
• Assignee: Unassigned
• Reporter: Yin Huai
• Created: 31/Jul/14 18:51
• Updated: 15/Apr/15 23:06
• Resolved: 15/Apr/15 23:06

Description

When you try the following snippet in hive/console.

val data = sc.parallelize(Seq("""{"key.number1": "value1", "key.number2": "value2"}"""))
jsonRDD(data).registerAsTable("jt")
hql("select `key.number1` from jt")

You will find the name of key.number1 cannot be resolved.

org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Unresolved attributes: 'key.number1, tree:
Project ['key.number1]
 LowerCaseSchema 
  Subquery jt
   SparkLogicalPlan (ExistingRdd [key.number1#8,key.number2#9], MappedRDD[17] at map at JsonRDD.scala:37)

Note that when we fix this we should also fix the qualifiedName functions for attributes and table names.

Issue Links

• duplicates: SPARK-6898 Special chars in column names is broken
• is related to: SPARK-2760 Caching tables from multiple databases does not work
• is related to: SPARK-2063 Creating a SchemaRDD via sql() does not correctly resolve nested types