Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1808

JaninoRelMetadataProvider loading cache might cause OOM error

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.15.0
    • Component/s: core
    • Labels:
      None

      Description

      JaninoRelMetadataProvider has a static handler cache with size increasing over time for long running HS2, ending up causing OOM errors.

        /** Cache of pre-generated handlers by provider and kind of metadata.
         * For the cache to be effective, providers should implement identity
         * correctly. */
        private static final LoadingCache<Key, MetadataHandler> HANDLERS =
            CacheBuilder.newBuilder().build(
                new CacheLoader<Key, MetadataHandler>() {
                  public MetadataHandler load(@Nonnull Key key) {
                    //noinspection unchecked
                    return load3(key.def, key.provider.handlers(key.def),
                        key.relClasses);
                  }
                });
      ...
        /** Key for the cache. */
        private static class Key {
          public final MetadataDef def;
          public final RelMetadataProvider provider;
          public final ImmutableList<Class<? extends RelNode>> relClasses;
      ...
      

      The lifecycle for providers is per query and we have multiple providers instantiated on the lifecycle of a query. The entries are retained in the cache even when query planning has finished.

        Attachments

        1. calcite-retained-strong-refs.png
          248 kB
          Jesus Camacho Rodriguez

          Activity

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              jvaria Jagruti Varia
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: