Description
A dag with 15k x 1000k vertex may auto-reduce to 15k x 1. And while the data size is appropriate for 1 task attempt, this results in an increase in task attempt message processing of 1000x.
This jira aims to reduce the message processing in the auto-reduced task while keeping the amount of message processing in the AM the same or less.
Attachments
Attachments
Issue Links
- blocks
-
TEZ-3361 Fetch Multiple Partitions from the Shuffle Handler
- Closed