Hive
  1. Hive
  2. HIVE-6572

Use shimmed version of hadoop conf names for mapred.{min,max}.split.size{.*}

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0, 0.14.0
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      HadoopShims has a method to fetch config parameters by name so that they return the appropriate config param name for the appropriate hadoop version. We need to be consistent about using these versions.

      For eg:. mapred.min.split.size is deprecated with hadoop 2.x, and is instead called mapreduce.input.fileinputformat.split.minsize .

      Also, there is a bug in Hadoop23Shims, Hadoop20SShims and Hadoop20Shims that defines MAPREDMINSPLITSIZEPERNODE as mapred.min.split.size.per.rack and MAPREDMINSPLITSIZEPERRACK as mapred.min.split.size.per.node. This is wrong and confusing.

      1. HIVE-6572.patch
        13 kB
        Sushanth Sowmyan

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Sushanth Sowmyan
              Reporter:
              Sushanth Sowmyan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development