[HIVE-25085] MetaStore Clients are being shared across different sessions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: HiveServer2
• Labels: pull-request-available
• Assignee: Steve Carlin
• Reporter: Steve Carlin
• Created: 02/May/21 22:36
• Updated: 17/Nov/22 08:48
• Resolved: 23/Jun/21 10:21

Description

The Hive object (and the underlying MetaStoreClient object) seems to be getting shared across different sessions.  While most operations work, there can be occasional glitches. 

One such noted glitch is that when session 1 ends, it closes the connection.  If session 2 then tries an operation, the first try will fail.  Normally this can proceed because the RetryingMetaStoreClient will re-establish a new connection, but in some operations, the retrying logic will not kick in (by design).

It seems there was an attempt to fix this issue in HIVE-20682.  However, this implementation seems to be flawed.  The HiveSessionImpl object creates a Hive object and makes sure all thread queries belonging to the same session will run with the same Hive object.  The flaw is that the initial Hive Object within HiveSessionImpl is created in thread local storage.  The thread being run at that moment is not session specific.  It belongs to a thread pool that happens to be handling this specific session.

 

Issue Links

• relates to: HIVE-20682 Async query execution can potentially fail if shared sessionHive is closed by master thread.
• links to: GitHub Pull Request #2238