[MAPREDUCE-2652] MR-279: Cannot run multiple NMs on a single node

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: mrv2
• Hadoop Flags: Reviewed
• Assignee: Robert Joseph Evans
• Reporter: Robert Joseph Evans
• Created: 07/Jul/11 13:53
• Updated: 15/Nov/11 00:50
• Resolved: 31/Aug/11 14:08

Description

Currently in MR-279 the Auxiliary services, like ShuffleHandler, have no way to communicate information back to the applications. Because of this the Map Reduce Application Master has hardcoded in a port of 8080 for shuffle. This prevents the configuration "mapreduce.shuffle.port" form ever being set to anything but 8080. The code should be updated to allow this information to be returned to the application master. Also the data needs to be persisted to the task log so that on restart the data is not lost.

Issue Links

• relates to: MAPREDUCE-2750 [MR-279] NodeManager should start its servers on ephemeral ports
• relates to: MAPREDUCE-2947 Sort fails on YARN+MR with lots of task failures