[CALCITE-2449] RexSimplify overhaul

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.17.0
• Component/s: core
• Assignee: Unassigned
• Reporter: Vladimir Sitnikov
• Created: 07/Aug/18 07:52
• Updated: 16/Apr/21 07:22

Description

I would like to update RexSimplify as follows:

1) Avoid multiple passes over the same expressions. Current logic calls "simplify" in multiple places, and it might cause simplification of the same tree again and again.

2) I would like to rework unknownAsFalse into support for IS TRUE. Current support for unknownAsFalse is hard to follow, and it looks like IS_TRUE(expr) allows for the same optimizations. In other words, "unknownAsFalse" would become a silent adding of IS_TRUE on top of the input expression. Then rules like OR(x, y) IS TRUE ==> x IS TRUE or y IS TRUE would push IS TRUE downstream.

3) I would like to "reorder" predicates in a defined order (e.g. rewrite 5=x ==> x=5 or (x=1 or y=2) and (z=3) ==> (z=3) and (x=1 or y=2). It looks like adding int nodeCount; to RexNode would be helpful to rank rex nodes.

4) There are other bugs identified by expression fuzzer (like and(null, not(null)) conversion to true and so on)

Issue Links

• is related to: CALCITE-2401 Improve RelMdPredicates performance
• is related to: CALCITE-4559 Create 'interface RexRule', a modular rewrite for row-expressions
• is related to: CALCITE-2338 Make Simplification API more conservative
• is related to: CALCITE-3178 RexSimplify.simplifyOrTerms slow with large OR filters

Sub-Tasks

• RexSimplify: reorder predicates to a canonical form as a part of RexSimplify: Closed
• RexSimplify: fuse unknownAsFalse and IS TRUE handling: Open
• RexSimplify: avoid multiple similar optimization passes over the same nodes: Open
• RexProgramTest: move "rex building" methods to base class: Closed
• Add randomized test for better code coverage of rex node create and simplification: Closed