Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.20.2
-
None
-
None
-
None
Description
This was an interesting one. Our rack awareness script contains a 1-1 mapping from host/ip to rack. We added a new rack's worth of machines without updating the awareness script, and when the script was called, it returned absolutely no results for the new machines.
This resulted in the surprising result that basically the entire cluster stopped working. Even tasks or blocks assigned to nodes with a valid rack seemed to fail. The errors were only detectable by looking in the namenode and jobtracker logs, making it take a while before we could figure out the problem. After fixing the rack awareness script, everything returned to normal operation.
It seems to me that either the error should be raised more aggressively, or a "default" rack should be assumed. This would keep simple mistakes from making the entire cluster unusable.
Attachments
Issue Links
- is related to
-
-
- Closed
-