[HIVE-16483] HoS should populate split related configurations to HiveConf

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Spark
• Assignee: Chao Sun
• Reporter: Chao Sun
• Created: 19/Apr/17 21:11
• Updated: 22/May/18 23:59
• Resolved: 24/Apr/17 16:57

Description

There are several split related configurations, such as MAPREDMINSPLITSIZE, MAPREDMINSPLITSIZEPERNODE, MAPREDMINSPLITSIZEPERRACK, etc., that should be populated to HiveConf. Currently we only do this for MAPREDMINSPLITSIZE.
All the others, if not set, will be using the default value, which is 1.

Without these, Spark sometimes will not merge small files for file formats such as text.