[SPARK-15852] Improve query planning performance for wide nested schema

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: SQL
• Target Version/s: 2.0.0
• Assignee: Eric Liang
• Reporter: Reynold Xin
• Created: 09/Jun/16 23:46
• Updated: 30/Jun/16 07:24
• Resolved: 10/Jun/16 07:16

Description

This tracks a list of issues to improve query planning (and code generation) performance for wide nested schema.

Issue Links

• relates to: SPARK-16320 Document G1 heap region's effect on spark 2.0 vs 1.6

Sub-Tasks

• Should truncate toString() of very wide schemas: Resolved
• Cache Metadata.hashCode and use a singleton for Metadata.empty: Resolved
• Replace inefficient foldLeft() call in PartitionStatistics: Resolved
• Reduce collections allocations in Catalyst tree transformation methods: Resolved
• Replace n^2 loop in BindReferences: Resolved
• Update microbenchmark results: Resolved