Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1979

"Output directory already exists" error in gridmix when gridmix.output.directory is not defined

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.22.0
    • contrib/gridmix
    • None
    • Reviewed

    Description

      "Output directory already exists" error is seen in gridmix when gridmix.output.directory is not defined. When gridmix.output.directory is not defined, then gridmix uses inputDir/gridmix/ as output path for gridmix run. Because gridmix is creating outputPath(in this case, inputDir/gridmix/) at the begining, the output path to generate-data-mapreduce-job(i.e. inputDir) already exists and becomes error from mapreduce.

      There is need for creation of this outputPath in any case(whether user specifies the path using gridmix.output.directory OR gridmix itself considering inputDir/gridmix/ ) even though the paths are automatically created for output paths of mapreduce jobs(like mkdir -p), because gridmix needs to set 777 permissions for this outputPath sothat different users can create different output directories of different mapreduce jobs within this gridmix run.

      The other case in which this problem is seen is when gridmix.output.directory is defined as a relative path. This is because in this case also, gridmix tries to create relative path under ioPath/ and thus the same issue.

      Attachments

        1. 1979.v1.patch
          5 kB
          Ravi Gummadi
        2. 1979.v1.3.patch
          7 kB
          Ravi Gummadi
        3. 1979.v1.2.patch
          5 kB
          Ravi Gummadi
        4. 1979.v1.1.patch
          5 kB
          Ravi Gummadi
        5. 1979.patch
          0.9 kB
          Ravi Gummadi

        Activity

          People

            ravidotg Ravi Gummadi
            ravidotg Ravi Gummadi
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: