Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3

Output directories are not cleaned up before the reduces run

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.1.0
    • 0.1.0
    • None
    • None

    Description

      The output directory for the reduces is not cleaned up and therefore if you can see left overs from previous runs, if they had more reduces. For example, if you run the application once with reduces=10 and then rerun with reduces=8, your output directory will have frag00000 to frag00009 with the first 8 fragments from the second run and the last 2 fragments from the first run.

      Attachments

        1. clean-out-dir.patch
          0.7 kB
          Owen O'Malley
        2. noclobber.patch
          2 kB
          Owen O'Malley

        Activity

          People

            omalley Owen O'Malley
            omalley Owen O'Malley
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: