[CALCITE-4029] ProjectRemoveRule auto pruning may prevent rules from running if mixed conventions are used in a logical plan

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 1.23.0
• Fix Version/s: 1.24.0
• Component/s: core
• Assignee: Unassigned
• Reporter: Anton Haidai
• Created: 28/May/20 22:39
• Updated: 28/May/21 10:12
• Resolved: 02/Jun/20 18:11

Description

Preconditions to reproduce the issue:

1. Logical plan has mixed conventions (for example, a bottom node is a TableScan in a final convention while other nodes are regular logical nodes with NONE convention).
2. There is a rule that expects a logical node with an input (like a rule matching "operand(LogicalSort.class, operand(RelNode.class, any()))")
3. A project over the scan is trivial (like SELECT * FROM ...)

The issue is related to https://issues.apache.org/jira/browse/CALCITE-3939, please see comments for a detailed debugging of a real-life reproducing case.

Example:

Logical plan with a leaf nodes in a custom convention:

LogicalSort[NONE]
 LogicalProject[NONE]
  CustomScan[CUSTOM_CONVENTION]

A rule configured (RuleX) matches "operand(LogicalSort.class, operand(RelNode.class, any()))".

Without ProjectRemoveRule auto pruning

ProjectRemoveRule recognizes LogicalProject as trivial an merges it into a single RelSet with CustomScan. 

RuleX can run on top of this change as far as LogicalProject has a logical node (LogicalProject in RelSubset[NONE]) as an input.

 

With ProjectRemoveRule auto pruning

ProjectRemoveRule recognizes LogicalProject as trivial but removes it with it's RelSet so the CustomScan is the only node in it's RelSet, RelSubset[CUSTOM_CONVENTION].

RuleX can't run on top of this change as far as LogicalProject has an empty input RelSubset[NONE] of the RelSet with the CustomScan.

Possible workarounds

1. Disable ProjectRemoveRule auto pruning.
2. Use only logical nodes in a logical plan, for the example above: use LogicalScan - >  CustomScanRule - > CustomScan instead of direct use of CustomScan.

Issue Links

• is caused by: CALCITE-3939 Change UnionEliminatorRule and ProjectRemoveRule to auto pruning SubstitutionRule