[SPARK-7713] Use shared broadcast hadoop conf for partitioned table scan.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: SQL
• Target Version/s: 1.4.0
• Assignee: Yin Huai
• Reporter: Yin Huai
• Created: 18/May/15 19:54
• Updated: 20/May/15 18:24
• Resolved: 20/May/15 18:24

Description

While debugging SPARK-7673, we also found that we are broadcasting a hadoop conf for every Partition (backed by a Hadoop RDD). It also causes the performance regression of compiling a query involving a large number of partitions.

Issue Links

• links to: [Github] Pull Request #6252 (yhuai)