[GOBBLIN-1524] Limit the number of container launch threads used by YarnService

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.15.0
• Fix Version/s: 0.17.0
• Component/s: gobblin-yarn
• Assignee: Sudarshan Vasudevan
• Reporter: Sudarshan Vasudevan
• Created: 23/Aug/21 16:51
• Updated: 14/Jun/23 04:13

Description

Currently, there are no limits on the number of threads used, which can result in OOMEs with "unable to create a new native thread". This PR makes the number of threads configurable with a default of 1024.