[SPARK-5354] Set InMemoryColumnarTableScan's outputPartitioning and outputOrdering

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: SQL
• Assignee: Nong Li
• Reporter: Yin Huai
• Created: 21/Jan/15 17:58
• Updated: 04/Nov/15 17:58
• Resolved: 03/Nov/15 03:19

Description

Updated
Right now, Spark SQL is not aware of the outputPartitioning and outputOrdering of a InMemoryColumnarTableScan. Actually, we can just inherit these two properties from the SparkPlan of the cached table.

Original
Right now, Spark SQL is not aware of the partitioning scheme of a leaf SparkPlan (e.g. an input table). So, even users want to re-partitioning the data in advance, Exchange operators will still be used.

Issue Links

• is duplicated by: SPARK-4849 Pass partitioning information (distribute by) to In-memory caching
• links to: [Github] Pull Request #9404 (nongli)