Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Currently tez.runtime.shuffle.memory-to-memory.enable is set to false by default. We need to evaluate the usefulness of this parameter and enable it by default if it provides good perf boosts.
There is also a possibility that waitForInMemoryMerge() will return in sub milliseconds when this parameter is enabled, causing pressure on network resources. Related JIRA: https://issues.apache.org/jira/browse/TEZ-1091