[CALCITE-3901] Introduce an option to disable metadata handler regeneration

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: core
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Liya Fan
• Created: 08/Apr/20 12:14
• Updated: 18/Mar/21 23:33

Description

According to our investigation, Calcite can produce large performance overhead due to meta data handler regeneration.

To illustrate, suppose we want to get the average size of a rel node, by calling the RelMetadataQuery#getAverageRowSize method. If the rel node type has not been registered before, JaninoRelMetadataProvider#revise method will be called, which will regerenate and recompile the code for the metadata handler.

This process is time-consuming, as compiling code by Janino is time-consuming: It takes more than 10ms to compile a single source file, even if the file is small. According to our investigation, performance overhead due to this problem can be over 10% of the total optimization time.

Therefore, we introduce an option to disable metadata handler regeneration. This forces the client to register all rel node types before calling the metadata service, so as to avoid performance being stolen silently.

For some scenarios, the client mistakenly believes they have registered all rel node types, and this option helps them find the types they have missed.

Issue Links

• is part of: CALCITE-4539 Support pluggable metadata handlers and caching
• links to: GitHub Pull Request #1901