Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25469

Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 3.0.0
    • Fix Version/s: 2.4.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Pattern matching is utilized for each call of Concat.eval, Reverse.nullSafeEval and ElementAt.nullSafeEval and thus for each record of a dataset. This could lead to a performance penalty when expression are executed in the interpreted mode. The goal of this ticket is to reduce usage of pattern matching within the mentioned "eval" methods.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mn-mikke Marek Novotny
                Reporter:
                mn-mikke Marek Novotny
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: