[MAPREDUCE-7276] hadoop job fast fail is not working properly

• Type: Bug
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.2.0, 2.9.2
• Component/s: applicationmaster
• Labels: pull-request-available

Environment

hadoop 2.9.2, hive 2.3.2 and hive 0.13

• Assignee: Wang Yan
• Reporter: Wang Yan
• Created: 20/Apr/20 05:40
• Updated: 13/Oct/21 18:36

Description

User operation : run a hive job with HFDS disk quota specified, and enable the fast fail feature to fast fail a job when hdfs disk quota limitation is exceeded (mapreduce.job.dfs.storage.capacity.kill-limit-exceed=true).

Expect behaviour : the job fails fast when one task attempt fails due to exceeding the HDFS quota limitation.

Actual behaviour : fast fail is not working at all. 

Cause of the bug : TaskAttemptImpl does not handle fast fail at all in the FAILED_FINISHING_TRANSITION event.

 
 

Issue Links

• links to: GitHub Pull Request #1971
• links to: GitHub Pull Request #1971