Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
Currently we have a naive way of computing the number of shuffle mergers required for a ShuffleMapStage by considering the resulting number of partitions. This needs to be further brainstormed and enhanced.