Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4389

Map only jobs should work even for MapOutput(Key|Value)Class configuration

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: job submission, mrv2
    • Labels:
      None

      Description

      I have written a Map only job and have set MapOutput(Key|Value)Class.
      Job was passing when I set TextOutputFormat, however, when I changed the OF to SequenceFileOutputFormat, job started failing.

      Later, I have set them to Output(Key|Value)Class, and job was passing.

      I think, for Map only jobs, jobs should pass even for MapOutput(Key|Value)Class.

        Activity

        Hide
        qwertymaniac Harsh J added a comment -

        Output formats always look at OutputK/V classes. The intermediate output writer always looks at MapOutputK/V classes, which is checked only if numReduces > 0. Hence, for all final outputs, be it in map stage or be it in reduce stage, its only the former that matters to the O/F.

        I'm for closing this JIRA as invalid, unless you feel a doc improvement at least needs to be made.

        Show
        qwertymaniac Harsh J added a comment - Output formats always look at OutputK/V classes. The intermediate output writer always looks at MapOutputK/V classes, which is checked only if numReduces > 0. Hence, for all final outputs, be it in map stage or be it in reduce stage, its only the former that matters to the O/F. I'm for closing this JIRA as invalid, unless you feel a doc improvement at least needs to be made.

          People

          • Assignee:
            kam_iitkgp Bhallamudi Venkata Siva Kamesh
            Reporter:
            kam_iitkgp Bhallamudi Venkata Siva Kamesh
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development