[HIVE-14925] MSCK repair table hang while running with multi threading enabled

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: CLI

Release Note

Issue: MSCK is failing in multithreaded execution

Solution:
  - Moved Path processor logic to an external class which will avoid code duplication and it will be used in both multi-threaded and single threaded execution.

• Assignee: Ratheesh Kamoor
• Reporter: Ratheesh Kamoor
• Created: 11/Oct/16 02:39
• Updated: 21/Oct/22 07:31

Description

MSCK REPAIR TABLE hanging while running with multi-threading enabled (default). I think it is because of a major design flaw in how thread pool implemented in HiveMetaSoreChecker class / checkPartitionDirs method. This method has a thread pool which register Callable but callable makes a recursive call to checkPartitionDirs method again. This code will hang when number of directories is more than thread pool size.