Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6572

Use shimmed version of hadoop conf names for mapred.{min,max}.split.size{.*}

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0, 0.14.0
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      HadoopShims has a method to fetch config parameters by name so that they return the appropriate config param name for the appropriate hadoop version. We need to be consistent about using these versions.

      For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead called mapreduce.input.fileinputformat.split.minsize .

      Also, there is a bug in Hadoop23Shims, Hadoop20SShims and Hadoop20Shims that defines MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong and confusing.

        Attachments

        1. HIVE-6572.patch
          13 kB
          Sushanth Sowmyan

          Issue Links

            Activity

              People

              • Assignee:
                sushanth Sushanth Sowmyan
                Reporter:
                sushanth Sushanth Sowmyan
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: