[HIVE-18808] Make compaction more robust when stats update fails

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Eugene Koifman Assign to me
• Reporter: Eugene Koifman
• Created: 27/Feb/18 00:10
• Updated: 02/Oct/19 01:06
• Resolved: 27/Feb/18 21:00

Description

 

Worker.gatherStats() runs a "analyze table..." command to update stats which requires SessionState.  SessionState objects are cached in ThreadLocal.  If for some reason Session init fails, it may still get attached to the thread which then causes a subsequent request that uses the same thread to gather stats fail because it has a bad session object.  HIVE-15658 describes the same issue in a different context.  

There is currently no way to recycle a session from outside HMS.

Failure to gather stats should not kill a compaction job which then prevents Cleaner from running.

Issue Links

• relates to: HIVE-15658 hive.ql.session.SessionState start() is not atomic, SessionState thread local variable can get into inconsistent state