[MAPREDUCE-6425] ShuffleHandler passes wrong "base" parameter to getMapOutputInfo if mapId is not in the cache.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: mrv2, nodemanager
• Target Version/s: 2.7.2
• Hadoop Flags: Reviewed
• Assignee: Zhihai Xu
• Reporter: Zhihai Xu
• Created: 03/Jul/15 01:42
• Updated: 06/Jan/17 01:40
• Resolved: 06/Jul/15 08:23

Description

ShuffleHandler passes wrong base parameter to getMapOutputInfo if mapId is not in the cache.
getMapOutputInfo expected the base parameter is getBaseLocation(jobId, user) + mapId
When it is called inside populateHeaders, the base parameter is set correctly

        String base = outputBaseStr + mapId;
        MapOutputInfo outputInfo = getMapOutputInfo(base, mapId, reduce, user);

When it is called outside populateHeaders, the base parameter is set wrongly to outputBasePathStr after number of mapId cached exceeds mapOutputMetaInfoCacheSize.

         String outputBasePathStr = getBaseLocation(jobId, user);
          MapOutputInfo info = mapOutputInfoMap.get(mapId);
          if (info == null) {
            info = getMapOutputInfo(outputBasePathStr, mapId, reduceId, user);
          }