Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4025

AM can crash if task attempt reports bogus progress value

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 0.23.2
    • Component/s: mr-am, mrv2
    • Labels:
      None

      Description

      If a task attempt reports a bogus progress value (e.g.: something above 1.0) then the AM can crash like this:

      java.lang.ArrayIndexOutOfBoundsException: 12
      	at org.apache.hadoop.mapred.PeriodicStatsAccumulator.extend(PeriodicStatsAccumulator.java:185)
      	at org.apache.hadoop.mapred.WrappedPeriodicStatsAccumulator.extend(WrappedPeriodicStatsAccumulator.java:31)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.updateProgressSplits(TaskAttemptImpl.java:1043)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.access$4100(TaskAttemptImpl.java:136)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$StatusUpdater.transition(TaskAttemptImpl.java:1509)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$StatusUpdater.transition(TaskAttemptImpl.java:1490)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
      	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
      	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:931)
      	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:135)
      	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:886)
      	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:878)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125)
      	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:74)
      	at java.lang.Thread.run(Thread.java:619)
      

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Jason Lowe
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development