Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8829

Improve expression performance

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • SQL
    • None

    Description

      This is an umbrella ticket for various improvements to DataFrame and SQL expression performance.

      These expressions can be found in the org.apache.spark.sql.catalyst.expressions package.

      Note that I also created a separate ticket for improving regular expression performance potentially using joni (SPARK-9213).

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8159 Improve expression function coverage (Spark 1.5)

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-9213 Improve regular expression performance (via joni)

          • Major - Major loss of function.
          • Resolved
          1.
          		 Create CodegenFallback and Unevaluable trait Sub-task Resolved Reynold Xin
          2.
          		 levenshtein directly on top of UTF8String Sub-task Resolved Tarek Auel
          3.
          		 cast from double/float to timestamp should not go through decimal Sub-task Resolved Wenchen Fan
          4.
          		 use double not decimal when cast double and float to timestamp Sub-task Resolved Wenchen Fan
          5.
          		 Implement code generation for all casts Sub-task Resolved Yijie Shen
          6.
          		 Implement code gen for Conv Sub-task Resolved Tarek Auel
          7.
          		 Implement code generation for Abs Sub-task Resolved L. C. Hsieh
          8.
          		 Implement code generation for Like and RLike Sub-task Resolved L. C. Hsieh
          9.
          		 Implement code generation for StringLPad and StringRPad Sub-task Resolved Tarek Auel
          10.
          		 Implement code generation for StringFormat Sub-task Resolved Tarek Auel
          11.
          		 Implement code generation for StringSpace Sub-task Resolved Tarek Auel
          12.
          		 Implement code generation for StringSplit Sub-task Resolved Tarek Auel
          13.
          		 Implement code generation for Substring Sub-task Resolved Tarek Auel
          14.
          		 Implement code generation for Ascii, Base64, and UnBase64 Sub-task Resolved Tarek Auel
          15.
          		 Implement code generation for Encode and Decode Sub-task Resolved Tarek Auel
          16.
          		 Implement code generation for FormatNumber Sub-task Resolved Tarek Auel
          17.
          		 Implement code generation for ScalaUDF Sub-task Resolved L. C. Hsieh
          18.
          		 Implement code generation for Conv Sub-task Resolved Tarek Auel
          19.
          		 Implement code generation for Hex and Unhex Sub-task Resolved Tarek Auel
          20.
          		 Implement code generation for CreateArray, CreateStruct, and CreateNamedStruct Sub-task Resolved Yijie Shen
          21.
          		 Reuse Calendar instance in WeekOfYear Sub-task Resolved Tarek Auel
          22.
          		 Implement code generation for StringLocate Sub-task Resolved Davies Liu
          23.
          		 Fully implement code generation for ConcatWs Sub-task Resolved Yijie Shen
          24.
          		 Remove CodegenFallback from Abs / FormatNumber Sub-task Resolved Reynold Xin
          25.
          		 Implement code generation for In / InSet Sub-task Resolved L. C. Hsieh

          Activity

            People

              rxin Reynold Xin
              rxin Reynold Xin
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: