Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7291

Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

      The following are the situations:

      1. total task(map/reduce) count is same as the running limit of task(mapreduce.job.running.map.limit/mapreduce.job.running.reduce.limit).
      2. And start job. -> And total tasks(map/reduce) are running. -> And failed attempt for some reasons.
      3. Request allocation of new containers because the attempt failed.
      4. Quickly receive allocation of new containers.
      5. However, new container is released because failed attempts have not been cleared up.(allocated == total == running limit)
      6. Subsequently, the failed attempts is terminated, but it is waiting forever.
      7. Job is stuck.

      We switched MR frameworks(2.7.1) and checked that it worked well.
      Perhaps it is related to MAPREDUCE-6697

      Can you help me?

       

        Attachments

        1. image-2020-08-10-15-13-03-930.png
          226 kB
          Lee young gon

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dasom Lee young gon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: