Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25860

Replace Literal(null, _) with FalseLiteral whenever possible

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Optimizer, SQL
    • Labels:
      None

      Description

      We should have a new optimization rule that replaces Literal(null, _) with FalseLiteral in conditions in Join and Filter, predicates in If, conditions in CaseWhen.

      The underlying idea is that those expressions evaluate to false if the underlying expression is null (as an example see GeneratePredicate$create or doGenCode and eval methods in If and CaseWhen). Therefore, we can replace Literal(null, _) with FalseLiteral, which can lead to more optimizations later on.

        Attachments

          Activity

            People

            • Assignee:
              aokolnychyi Anton Okolnychyi
              Reporter:
              aokolnychyi Anton Okolnychyi
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: