Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9843

Catalyst: Allow adding custom optimizers

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.1
    • 2.0.0
    • SQL
    • None

    Description

      Currently there's only option to plug into query planning. This provides limited capability for optimizing queries.

      Allowing specifying custom optimizers would allow applications that make use of spark sql tune their workflows. One example would be inserting repartitions into query plan which isn't generally applicable but if you know range of possible queries you can do that on the backend.

      Attachments

        Activity

          People

            robert3005 Robert Kruszewski
            robert3005 Robert Kruszewski
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: