Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.22.0
-
None
-
None
-
Reviewed
Description
It would be nice that we can separately assign rack locality wait time.
In our use case, we would set node locality wait to zero and wait only rack locality.
I propose that we add two parameters
mapred.fairscheduler.locality.delay.nodetorack
mapred.fairscheduler.locality.delay.racktoany
This allows specifying the wait time on each stage.
And we can use
mapred.fairscheduler.locality.delay
as the default value of the above fields so that this is backward compatible.
Thoughts?