[CALCITE-4539] Support pluggable metadata handlers and caching

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: core
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: James Starr
• Created: 16/Mar/21 22:34
• Updated: 16/Feb/22 02:36

Description

Calcite janino backed metadata does not support all the functionality that the java reflection backed metadata system, such as customizing the cache to use soft refs or have a max size.  Furthermore, the janino backed metadata provider has many opinion that other are not universally shared, such as lazy loading, requiring all rels to registered to a global singleton, use of thread locals.  There is no easy way to currently override these opinions.

Calcites metadata system should consist of 3 APIs: metadata consumer, metadata implementors, metadata behavior.  The metadata consumer need to able to fetch the appropriate metadata and invalidate the cache.  Currently this is all done through RelMetadataQuery with some very leaky abstractions.  Metadata implementors create functions that extract the metadata for particular node types.  These functions are registered via ReflectiveRelMetadataProvider and ChainedRelMetadataProvider.  Attempting to configure the internal behavior does not currently have an api.  Currently portions of it a scattered through out various class and are frequently not extensible.  These could all be concentrated in a single class MetadataHandlerProvider, to allow for configuration of eager/lazy loading behavior, caching policy(thread safe, max size, soft references), or a clear api for some to go their own way for handler generation. 

 

Issue Links

• blocks: CALCITE-4786 Facilitate use of graalvm native-image compilation
• incorporates: CALCITE-3901 Introduce an option to disable metadata handler regeneration
• links to: GitHub Pull Request #2374
• links to: GitHub Pull Request #2378