Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.2.0
-
None
-
None
-
None
Description
When the queue has node-labels, the delays involved in waiting for node locality are a net-loss to the query.
Asking for locality in a node which is not available to the current queue is slowing down YARN task allocations.
Also specifically, for the 3-replica HDFS case, picking the replica that belongs within the node-label is more efficient than picking a non-available node & falling back to rack_local instead.