[CALCITE-1808] JaninoRelMetadataProvider loading cache might cause OOM error

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: core
• Assignee: Julian Hyde
• Reporter: Jagruti Varia
• Created: 26/May/17 10:46
• Updated: 27/Feb/24 22:23
• Resolved: 07/Dec/17 09:28

Description

JaninoRelMetadataProvider has a static handler cache with size increasing over time for long running HS2, ending up causing OOM errors.

  /** Cache of pre-generated handlers by provider and kind of metadata.
   * For the cache to be effective, providers should implement identity
   * correctly. */
  private static final LoadingCache<Key, MetadataHandler> HANDLERS =
      CacheBuilder.newBuilder().build(
          new CacheLoader<Key, MetadataHandler>() {
            public MetadataHandler load(@Nonnull Key key) {
              //noinspection unchecked
              return load3(key.def, key.provider.handlers(key.def),
                  key.relClasses);
            }
          });
...
  /** Key for the cache. */
  private static class Key {
    public final MetadataDef def;
    public final RelMetadataProvider provider;
    public final ImmutableList<Class<? extends RelNode>> relClasses;
...

The lifecycle for providers is per query and we have multiple providers instantiated on the lifecycle of a query. The entries are retained in the cache even when query planning has finished.

Issue Links

• relates to: HIVE-25472 Prevent hive-server2 from getting OOM(Compressed class space) (Backport HIVE-18920)