Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1095

Enable tez.runtime.shuffle.memory-to-memory.enable for mem-to-mem merging

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Currently tez.runtime.shuffle.memory-to-memory.enable is set to false by default. We need to evaluate the usefulness of this parameter and enable it by default if it provides good perf boosts.

      There is also a possibility that waitForInMemoryMerge() will return in sub milliseconds when this parameter is enabled, causing pressure on network resources. Related JIRA: https://issues.apache.org/jira/browse/TEZ-1091

      Attachments

        Activity

          People

            Unassigned Unassigned
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: