[CALCITE-794] Detect cycles when computing statistics

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Julian Hyde
• Reporter: Julian Hyde
• Created: 11/Jul/15 22:37
• Updated: 09/Jul/24 01:24
• Resolved: 11/Jan/16 04:54

Description

The graph of RelNodes is allowed to be cyclic. This causes problems when evaluating certain metadata, for example RelMetataQuery.areColumnsUnique. While computing the value for RelNode r, it might recurse through say a Project and hit r again. This causes a stack overflow.

We solve this by adding a map or set of active RelNodes. The map is stored within RelMetadataQuery, which can now be instantiated, and its methods are no longer static. The first call should instantiate a RelMetadataQuery, but all subsequent calls for metadata (perhaps several kinds of metadata) will use the same RelMetadataQuery instance, hence the same map.

Also add a RelMetadataQuery argument to the static "handler" methods in RelMdColumnUniqueness and similar classes.

This is a breaking change for people who have written a metadata handler, and might be subtle to detect, because the methods are invoked via reflection.

For code that is just using RelMetadataQuery methods, the change is still breaking, but the break points and remedy will be obvious: the methods are no longer static, so they need to change RelMetadataQuery.foo() to RelMetadataQuery.instance().foo().

Issue Links

• is depended upon by: CALCITE-1048 Make metadata more robust
• is depended upon by: CALCITE-751 Push aggregate with aggregate functions through join
• is related to: CALCITE-790 Remove relational expressions that reference themselves
• is related to: HIVE-12508 HiveAggregateJoinTransposeRule places a heavy load on the metadata system
• relates to: CALCITE-2057 StackOverflowError when running a JDBC query
• relates to: CALCITE-604 Invoke metadata providers via a class generated at runtime, rather than reflection
• relates to: CALCITE-1018 SortJoinTransposeRule not firing due to getMaxRowCount(RelSubset) returning null
• relates to: CALCITE-6363 Introduce a rule to derive more filters from inner join condition
• relates to: HIVE-12839 Upgrade Hive to Calcite 1.6