Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25860

Replace Literal(null, _) with FalseLiteral whenever possible

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Optimizer, SQL
    • None

    Description

      We should have a new optimization rule that replaces Literal(null, _) with FalseLiteral in conditions in Join and Filter, predicates in If, conditions in CaseWhen.

      The underlying idea is that those expressions evaluate to false if the underlying expression is null (as an example see GeneratePredicate$create or doGenCode and eval methods in If and CaseWhen). Therefore, we can replace Literal(null, _) with FalseLiteral, which can lead to more optimizations later on.

      Attachments

        Activity

          People

            aokolnychyi Anton Okolnychyi
            aokolnychyi Anton Okolnychyi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: