[HIVE-18513] Query results caching

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Query Planning
• Assignee: Jason Dere
• Reporter: Jason Dere
• Created: 22/Jan/18 23:31
• Updated: 02/Oct/24 11:44
• Resolved: 07/Feb/18 21:51

Description

Add a query results cache that can save the results of an executed Hive query for reuse on subsequent queries. This may be useful in cases where the same query is issued many times, since Hive can return back the results of a cached query rather than having to execute the full query on the cluster.

Issue Links

• is related to: HIVE-28551 Stale results when executing queries over recreated transactional tables
• relates to: HIVE-20834 Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query
• links to: Design doc
• links to: https://reviews.apache.org/r/65304/

Sub-Tasks

• Results cache invalidation based on ACID table updates: Closed
• Query results cache: Allow queries to refer to the pending results of a query that has not finished yet: Closed
• Fix unit test TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]: Closed
• Separate FetchWork required for each query that uses the results cache: Closed
• Metrics for results cache: Closed
• Error in results cache when query has identifiers with spaces: Closed
• query result cache interferes with explain analyze: Closed
• Concurrency fixes in QueryResultsCache: Closed
• Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails: Closed
• Poll notification events to invalidate the results cache: Closed
• Disable query results cache for for HiveServer2 doAs=true: Closed
• Query results cache: Improve ability of queries to use pending query results: Closed
• Option to allow external tables to use query results cache: Open
• Cleanup of results cache directory: Patch Available