[REEF-589] REEF crashes when new nodes are added to the clusters dynamically

• Type: Task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.13
• Assignee: Ignacio Cano
• Reporter: Ignacio Cano
• Created: 13/Aug/15 16:33
• Updated: 14/Aug/15 01:45
• Resolved: 14/Aug/15 01:45

Description

When trying to use REEF with Federation, we found a problem in REEF resource catalog.
If an admin happens to add a new node to the cluster dynamically, and future allocations are done in that node, REEF crashes as it is not able to find that node in the catalog.
Though we found this problem using YARN, it will happen the same for other RMs.

Issue Links

• depends upon: REEF-337 Support REEF on YARN Federation
• duplicates: REEF-568 Work around the federated YARN node reports problem
• is a clone of: REEF-568 Work around the federated YARN node reports problem
• is related to: YARN-2915 Enable YARN RM scale out via federation using multiple RM's