[SPARK-9925] Set SQLConf.SHUFFLE_PARTITIONS.key correctly for tests

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Sprint: Spark 1.5 doc/QA sprint
• Assignee: Yin Huai
• Reporter: Yin Huai
• Created: 13/Aug/15 00:39
• Updated: 05/Sep/15 01:58
• Resolved: 05/Sep/15 01:58

Description

Right now, in our TestSQLContext/TestHiveContext, we use override def numShufflePartitions: Int = this.getConf(SQLConf.SHUFFLE_PARTITIONS, 5) to set SHUFFLE_PARTITIONS. However, we never put it to SQLConf. So, after we use withSQLConf(SQLConf.SHUFFLE_PARTITIONS.key -> "number"), the number of shuffle partitions will be set back to 200.

Issue Links

• links to: [Github] Pull Request #8155 (yhuai)
• links to: [Github] Pull Request #8602 (davies)