Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5911

Terasort TeraOutputFormat does not check for output directory existance

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: examples
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The enforcement that the directory must not yet exist is implemented in FileOutputFormat#checkOutputSpecs by throwing FileAlreadyExistsException. However, terasort uses a specialized output format, TeraOutputFormat, which is a subclass of FileOutputFormat. The subclass overrides checkOutputSpecs, but does not re-implement the existence check and throw FileAlreadyExistsException.

      1. HADOOP-5911.patch
        1 kB
        Bruno P. Kinoshita

        Issue Links

          Activity

          Ivan Mitic created issue -
          Chris Nauroth made changes -
          Field Original Value New Value
          Description The enforcement that the directory must not yet exist is implemented in {{FileOutputFormat#checkOutputSpecs}} by throwing {{FileAlreadyExistsException}}. However, terasort uses a specialized output format, {{TeraOutputFormat}}, which is a subclass of {{FileOutputFormat}}. The subclass overrides {{checkOutputSpecs}}, but does not re-implement the existence check and throw {{FileAlreadyExistsException}}.
          Bruno P. Kinoshita made changes -
          Attachment HADOOP-5911.patch [ 12667588 ]
          Ivan Mitic made changes -
          Assignee Ivan Mitic [ ivanmi ] Bruno P. Kinoshita [ kinow ]
          Ivan Mitic made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Gera Shegalov made changes -
          Link This issue duplicates MAPREDUCE-4879 [ MAPREDUCE-4879 ]
          Ivan Mitic made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.6.0 [ 12327180 ]
          Resolution Fixed [ 1 ]
          Ivan Mitic made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Ivan Mitic made changes -
          Fix Version/s 2.6.0 [ 12327180 ]
          Ivan Mitic made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]

            People

            • Assignee:
              Bruno P. Kinoshita
              Reporter:
              Ivan Mitic
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development