[SPARK-6023] ParquetConversions fails to replace the destination MetastoreRelation of an InsertIntoTable node to ParquetRelation2

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: SQL
• Target Version/s: 1.3.0
• Assignee: Yin Huai
• Reporter: Yin Huai
• Created: 26/Feb/15 02:02
• Updated: 24/Apr/15 00:43
• Resolved: 26/Feb/15 14:42

Description

import sqlContext._
sql("drop table if exists test")

val df1 = sqlContext.jsonRDD(sc.parallelize((1 to 10).map(i => s"""{"a":$i}""")))
df1.registerTempTable("jt")
sql("create table test (a bigint) stored as parquet ")

sql("explain insert into table test select a from jt").collect.foreach(println)

The plan will be

[== Physical Plan ==]
[InsertIntoHiveTable (MetastoreRelation default, test, None), Map(), false]
[ PhysicalRDD [a#34L], MapPartitionsRDD[17] at map at JsonRDD.scala:41]

However, the write path should be converted to our own data source path.

Issue Links

• links to: [Github] Pull Request #4782 (yhuai)