Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5850

map/reduce doesn't run jobs with 0 maps

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, the framework ignores jobs that have 0 maps. This is incorrect. Many pipelines need the job to run (if nothing else, to create the output directory!) so that subsequent jobs don't fail. Effectively, there will be no map tasks and the reduce tasks should immediately set up the Reducer and RecordWriter and then call close on both since there are no inputs to the reduce. I believe it should just work if we remove the check...

        Attachments

        1. screenshot-1.jpg
          174 kB
          Ramya Sunil
        2. HADOOP-5850-20090522-branch-20-final.txt
          20 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-5850-20090522.txt
          21 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-5850-20090520-svn-branch-20.v2.txt
          18 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-5850-20090520-svn.1.txt
          18 kB
          Vinod Kumar Vavilapalli
        6. HADOOP-5850-20090519.txt
          14 kB
          Vinod Kumar Vavilapalli
        7. HADOOP-5850-20090519.1.txt
          14 kB
          Vinod Kumar Vavilapalli

          Activity

            People

            • Assignee:
              vinodkv Vinod Kumar Vavilapalli
              Reporter:
              owen.omalley Owen O'Malley
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: