Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6354

Consider using Guava LoadingCache to cache metadata objects opposed to a ConcurrentHashMap

    XMLWordPrintableJSON

    Details

    • Epic Color:
      ghx-label-9

      Description

      Look into replacing the ConcurrentHashMap(s) used in the Catalog service to cache metadata with LoadingCache.

      Caching metadata using a LoadingCache will allow:

      • Better coherency by adding expireAfter and refreshAfter clause to the cache
      • Weak reference eviction to respond to garbage collection
      • Putting a cap on maximum number of caches entires to avoid OOMs and excessive GC
      • Assigning different weights for each entry (For more efficient eviction)

      https://github.com/google/guava/wiki/CachesExplained
      https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/CacheBuilder.html
      https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html

        Attachments

          Activity

            People

            • Assignee:
              tianyiwang Tianyi Wang
              Reporter:
              mmokhtar Mostafa Mokhtar
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: