Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7164

FileOutputCommitter does not report progress while merging paths.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.3, 2.8.5, 2.9.2
    • 3.0.4, 3.1.2, 3.3.0, 3.2.1
    • None
    • None
    • Reviewed

    Description

      In cases where the rename and merge path logic takes more time than usual, the committer does not report progress and can cause job failure. This behavior was not present in Hadoop 1.x. This JIRA will fix it so that the old behavior for 1.x is restored.

      Attachments

        1. MAPREDUCE-7164.001.patch
          4 kB
          Kuhu Shukla
        2. MAPREDUCE-7164.002.patch
          7 kB
          Kuhu Shukla

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HADOOP-15961 S3A committers: make sure there's regular progress() calls

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              kshukla Kuhu Shukla
              kshukla Kuhu Shukla
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: