[SPARK-5603] Preinsert casting and renaming rule is needed in the Analyzer

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: SQL
• Target Version/s: 1.3.0
• Sprint: Spark 1.5 release
• Assignee: Yin Huai
• Reporter: Yin Huai
• Created: 04/Feb/15 23:02
• Updated: 12/Feb/15 04:41
• Resolved: 06/Feb/15 20:39

Description

For an INSERT INTO/OVERWRITE statement, we should add necessary Cast and Alias to the output of the query.

CREATE TEMPORARY TABLE jsonTable (a int, b string)
USING org.apache.spark.sql.json.DefaultSource
OPTIONS (
  path '...'
)
INSERT OVERWRITE TABLE jsonTable SELECT a * 2, a * 4 FROM table

For a*2, we should create an Alias, so the InsertableRelation can know it is the column a. For a*4, it is actually the column b in jsonTable. We should first cast it to StringType and add an Alias b to it.

Issue Links

• links to: [Github] Pull Request #4373 (yhuai)