Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-46173

Skipping trimAll call in stringToDate functions to avoid needless string copy

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 4.0.0
    • SQL

    Description

      In StringToDate function call we currently first call trimAll to remove any whitespace and isocontrol characters. Trimming results in copying the input string which is not really needed given that we can do all the parsing in place by just skipping the whitespace/isocontrol chars.

      Given that we have customers complaining about speed of stringtodate function, especially when input string is long/potentially malformed proposal is to skip trimAll call and do parsing in place.

      Attachments

        Issue Links

          Activity

            People

              dbatomic Aleksandar Tomic
              dbatomic Aleksandar Tomic
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: