Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.2.0
-
None
-
None
Description
Currently BaseSemanticAnalyzer keeps a copy of thread-local Hive object to connect to HMS. However, in some cases Hive may overwrite the existing Hive object:
Hive#getInternal:
private static Hive getInternal(HiveConf c, boolean needsRefresh, boolean isFastCheck, boolean doRegisterAllFns) throws HiveException { Hive db = hiveDB.get(); if (db == null || !db.isCurrentUserOwner() || needsRefresh || (c != null && db.metaStoreClient != null && !isCompatible(db, c, isFastCheck))) { return create(c, false, db, doRegisterAllFns); } if (c != null) { db.conf = c; } return db; }
This poses an potential problem: if one first instantiates a BaseSemanticAnalyzer object with the current Hive object (let's call it A), and for some reason A is overwritten by B with the code above, then BaseSemanticAnalyzer may keep using A to contact HMS, which will leak connections.
This can be reproduced by the following steps:
1. open a session
2. execute some simple query such as desc formatted src
3. change a metastore property (I know, this is not a perfect example...), for instance: set hive.txn.timeout=500
4. run another command such as desc formatted src again
Notice that in step 4), since a metavar is changed the isCompatible will return false, and hence a new Hive object is created. As result, you'll observe in the HS2 log that an connection has been leaked.