Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6864

Hadoop streaming creates 2 mappers when the input has only one block

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:

      Description

      If a streaming job is run against input that is less than 2 blocks, 2 mappers will be created, both operating on the same split, both producing (duplicate) output. In some cases the second mapper will consistently fail. I've not seen the failure with input less than 10 bytes or more than a couple MB. I have seen it with a 4kB input.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              templedf Daniel Templeton
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: