Hadoop Common
  1. Hadoop Common
  2. HADOOP-5850

map/reduce doesn't run jobs with 0 maps

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, the framework ignores jobs that have 0 maps. This is incorrect. Many pipelines need the job to run (if nothing else, to create the output directory!) so that subsequent jobs don't fail. Effectively, there will be no map tasks and the reduce tasks should immediately set up the Reducer and RecordWriter and then call close on both since there are no inputs to the reduce. I believe it should just work if we remove the check...

      1. HADOOP-5850-20090522-branch-20-final.txt
        20 kB
        Vinod Kumar Vavilapalli
      2. HADOOP-5850-20090522.txt
        21 kB
        Vinod Kumar Vavilapalli
      3. screenshot-1.jpg
        174 kB
        Ramya Sunil
      4. HADOOP-5850-20090520-svn-branch-20.v2.txt
        18 kB
        Vinod Kumar Vavilapalli
      5. HADOOP-5850-20090520-svn.1.txt
        18 kB
        Vinod Kumar Vavilapalli
      6. HADOOP-5850-20090519.1.txt
        14 kB
        Vinod Kumar Vavilapalli
      7. HADOOP-5850-20090519.txt
        14 kB
        Vinod Kumar Vavilapalli

        Activity

        Owen O'Malley created issue -
        Vinod Kumar Vavilapalli made changes -
        Field Original Value New Value
        Assignee Vinod K V [ vinodkv ]
        Vinod Kumar Vavilapalli made changes -
        Attachment HADOOP-5850-20090519.txt [ 12408448 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment HADOOP-5850-20090519.1.txt [ 12408469 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment HADOOP-5850-20090520-svn.1.txt [ 12408613 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Ramya Sunil made changes -
        Attachment screenshot-1.jpg [ 12408771 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment HADOOP-5850-20090522.txt [ 12408793 ]
        Vinod Kumar Vavilapalli made changes -
        Devaraj Das made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Owen O'Malley
          • Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development