It seems there is a bug on FSAppAttempt#getAllowedLocalityLevelByTime method
If you first invoke this method, it doesn't init time in lastScheduledContainer and this will lead to execute these code for next invokation:
the waitTime will subtract to FsApp startTime, and this will be easily more than the delay time and allowedLocality degrade. Because FsApp startTime will be start earlier than currentTimeMs. So we should add the initial time of priority to prevent comparing with FsApp startTime and allowedLocalityLevel degrade. And this problem will have more negative influence for small-jobs. The YARN-4399 also discuss some problem in aspect of locality.