[HIVE-6572] Use shimmed version of hadoop conf names for mapred.{min,max}.split.size{.*}

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0, 0.14.0
• Fix Version/s: 0.13.0
• Assignee: Sushanth Sowmyan
• Reporter: Sushanth Sowmyan
• Created: 06/Mar/14 23:09
• Updated: 20/Mar/14 22:01
• Resolved: 12/Mar/14 22:54

Description

HadoopShims has a method to fetch config parameters by name so that they return the appropriate config param name for the appropriate hadoop version. We need to be consistent about using these versions.

For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead called mapreduce.input.fileinputformat.split.minsize .

Also, there is a bug in Hadoop23Shims, Hadoop20SShims and Hadoop20Shims that defines MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong and confusing.

Issue Links

• is duplicated by: HIVE-5910 In HiveConf, the name of mapred.min.split.size.per.rack is MAPREDMINSPLITSIZEPERNODE and the name of mapred.min.split.size.per.node is MAPREDMINSPLITSIZEPERRACK