Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6592

MR ApplicationMaster is not supposed to resolve rack locality! It should ask RM or NN about the topology

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      While working on configuring Rack Awareness (following HDP Documentation), we realized that this does not work for MR applications, unless the rack topology resolution script is installed on the node managers where the application masters reside. Otherwise, the AM will thrown an IOException: Cannot run program "/etc/hadoop/conf/rack_topology.sh".

      This is contradictory to the documentations which states it makes sense to have the scripts that resolve the rack and node names on the Resource Manager and Name Node.

      It is clearly not scalable to require to install these scripts on all the node managers!

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                babakbehzad Babak Behzad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: