Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12949

Support common expression elimination

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: SQL
    • Labels:
      None

      Issue Links

        Activity

        Hide
        davies Davies Liu added a comment -

        After some prototype, enable common expression elimination could have 10+% improvement on stddev, but 50% regression on Kurtosis, have not figure why, maybe JIT can already eliminate the common expressions (given the fact that Kurtosis is only 20% slower than stddev)? If yes, we may not want to do this.

        Show
        davies Davies Liu added a comment - After some prototype, enable common expression elimination could have 10+% improvement on stddev, but 50% regression on Kurtosis, have not figure why, maybe JIT can already eliminate the common expressions (given the fact that Kurtosis is only 20% slower than stddev)? If yes, we may not want to do this.
        Hide
        davies Davies Liu added a comment -

        Common subexpress elimination in Aggregate are supported by SPARK-14951, so close this.

        Show
        davies Davies Liu added a comment - Common subexpress elimination in Aggregate are supported by SPARK-14951 , so close this.

          People

          • Assignee:
            viirya Liang-Chi Hsieh
            Reporter:
            davies Davies Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development