[SPARK-36665] Add more Not operator optimizations

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: SQL
• Assignee: Kazuyuki Tanimura
• Reporter: Kazuyuki Tanimura
• Created: 03/Sep/21 22:49
• Updated: 08/Feb/22 00:57
• Resolved: 07/Nov/21 08:18

Description

BooleanSimplification should be able to do more simplifications for Not operators applying following rules

1. Not(null) == null
   1. e.g. IsNull(Not(...)) can be IsNull(...)
2. (Not(a) = b) == (a = Not(b))
   1. e.g. Not(...) = true can be (...) = false
3. (a != b) == (a = Not(b))
   1. e.g. (...) != true can be (...) = false

Issue Links

• relates to: SPARK-38132 Remove NotPropagation
• links to: [Github] Pull Request #33930 (kazuyukitanimura)
• links to: [Github] Pull Request #35400 (kazuyukitanimura)
• links to: [Github] Pull Request #35400 (kazuyukitanimura)
• links to: [Github] Pull Request #35428 (kazuyukitanimura)