[HIVE-23737] LLAP: Reuse dagDelete Feature Of Tez Custom Shuffle Handler Instead Of LLAP's dagDelete

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: llap
• Labels: pull-request-available
• Assignee: Syed Shameerur Rahman
• Reporter: Syed Shameerur Rahman
• Created: 22/Jun/20 08:24
• Updated: 22/Jan/21 09:57

Description

LLAP have a dagDelete feature added as part of HIVE-9911, But now that Tez have added support for dagDelete in custom shuffle handler (TEZ-3362) we could re-use that feature in LLAP.
There are some added advantages of using Tez's dagDelete feature rather than the current LLAP's dagDelete feature.

1) We can easily extend this feature to accommodate the upcoming features such as vertex and failed task attempt shuffle data clean up. Refer TEZ-3363 and TEZ-4129

2) It will be more easier to maintain this feature by separating it out from the Hive's code path.

Issue Links

• links to: GitHub Pull Request #1195