Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1461

Support Short-circuit Expression Evaluation

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.1, 1.1.0
    • SQL
    • None

    Description

      Short-circuit expression evaluation impacts the performance significantly.

      e.g.
      In expression: (a>b) && (c>d), if a>b equals false or null, the expression value is false or null definitely without considering the value of sub expression c>d.

      However, if c or d contains a stateful UDF (for example, the UDF row_sequence) as its child expression, we have to evaluate the stateful expression but ignore its result.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            chenghao Cheng Hao
            chenghao Cheng Hao
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment