[ATLAS-683] Refactor local type-system cache with cache provider interface

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.7-incubating
• Fix Version/s: 0.7-incubating
• Labels: high-availability performance scalability
• Flags: Patch
• Assignee: venkata madugundu
• Reporter: venkata madugundu
• Created: 18/Apr/16 11:39
• Updated: 25/May/16 13:23
• Resolved: 20/May/16 06:17

Description

As noted in ATLAS-488, local type-system cache makes Atlas runtime stateful and prevents multiple Atlas instances to be active in a cluster. Either the type-cache should be synched across Atlas instances (on all type create/update requests) or the type-cache should be moved out of Atlas to something like a distributed cache.

1. As a first step, the local type-cache code in TypeSystem.java can be refactored to be carved out as an interface like TypeCacheProvider (whose default implementation for a standalone Atlas server would just use in-process local cache). The cache provider implementation itself could be specified as an optional configuration property. Expert users of Atlas can choose to inject a custom cache provider which can likely hit a distributed cache. We are evaluating the use of a distributed cache.

2. As a second step, some more refactoring can be done to minimize/optimize the calls made to TypeSystem for type lookup queries. Essentially, in a given transaction/request, once a type lookup is done, it should not be requeried again. A request scoped variable (guice would probably help with that scoping) can hold all the lookups made in a request. This might sound like a cache of a cache, but I think it should help in reducing the hits to cache provider if the provider is hitting a remote cache.

Issue Links

• relates to: ATLAS-825 Redis-based implementation of type cache provider
• links to: Review Board