[GOBBLIN-1661] Make Yarn container and helix instance allocation group by tag

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: gobblin-helix, gobblin-yarn
• Assignee: Abhishek Tiwari
• Reporter: Hanghang Liu
• Created: 08/Jun/22 23:26
• Updated: 22/Jun/22 22:10
• Resolved: 22/Jun/22 22:10

Description

This change enable Helix instances use the specific tag defined(if exists) in their Job/Workflow. This makes the Helix cluster management easier and more strait forward, as each job's instances can carry unique Helix tag. 

In this case, for each Gobblin job run, helix tasks within this job will
be assigned to containers with desired helix tag and resource. This can make within one Gobblin job run, multiple resource config can be set for different topics, instead of the need of creating new Helix cluster for different Gobblin job run setting.