Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5850

map/reduce doesn't run jobs with 0 maps

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 0.20.0
    • 0.20.1
    • None
    • None
    • Reviewed

    Description

      Currently, the framework ignores jobs that have 0 maps. This is incorrect. Many pipelines need the job to run (if nothing else, to create the output directory!) so that subsequent jobs don't fail. Effectively, there will be no map tasks and the reduce tasks should immediately set up the Reducer and RecordWriter and then call close on both since there are no inputs to the reduce. I believe it should just work if we remove the check...

      Attachments

        1. HADOOP-5850-20090519.txt
          14 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-5850-20090519.1.txt
          14 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-5850-20090520-svn.1.txt
          18 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-5850-20090520-svn-branch-20.v2.txt
          18 kB
          Vinod Kumar Vavilapalli
        5. screenshot-1.jpg
          174 kB
          Ramya Sunil
        6. HADOOP-5850-20090522.txt
          21 kB
          Vinod Kumar Vavilapalli
        7. HADOOP-5850-20090522-branch-20-final.txt
          20 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            omalley Owen O'Malley
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: