Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9843

Catalyst: Allow adding custom optimizers

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.1
    • Fix Version/s: 2.0.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Currently there's only option to plug into query planning. This provides limited capability for optimizing queries.

      Allowing specifying custom optimizers would allow applications that make use of spark sql tune their workflows. One example would be inserting repartitions into query plan which isn't generally applicable but if you know range of possible queries you can do that on the backend.

        Attachments

          Activity

            People

            • Assignee:
              robert3005 Robert Kruszewski
              Reporter:
              robert3005 Robert Kruszewski
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: