Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6892

Issues with the count of failed/killed tasks in the jhist file

    Details

    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      This adds some new job counters, the number of failed MAP/REDUCE tasks and the number of killed MAP/REDUCE tasks.

      Description

      Recently we encountered some issues with the value of failed tasks. After parsing the jhist file, JobInfo.getFailedMaps() returned 0, but actually there were failures.

      Another minor thing is that you cannot get the number of killed tasks (although this can be calculated).

      The root cause is that JobUnsuccessfulCompletionEvent contains only the successful map/reduce task counts. Number of failed (or killed) tasks are not stored.

        Attachments

        1. MAPREDUCE-6892-001.patch
          30 kB
          Peter Bacsko
        2. MAPREDUCE-6892-002.PATCH
          38 kB
          Peter Bacsko
        3. MAPREDUCE-6892-003.patch
          67 kB
          Peter Bacsko
        4. MAPREDUCE-6892-004.patch
          68 kB
          Peter Bacsko
        5. MAPREDUCE-6892-005.patch
          70 kB
          Peter Bacsko
        6. MAPREDUCE-6892-006.patch
          70 kB
          Peter Bacsko

          Activity

            People

            • Assignee:
              pbacsko Peter Bacsko
              Reporter:
              pbacsko Peter Bacsko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: