Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6377

Set the number of map output partitions for Exchange operator automatically based on the size of input tables and the reduce-side operation.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      It will be helpful to automatically set the number of shuffle partitions based on the size of input tables and the operation at the reduce side for an Exchange operator.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                yhuai Yin Huai
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: