Trying to run application on Nodelabel partition I found that the application execution time is delayed by 5 – 10 min for 500 containers . Total 3 machines 2 machines were in same partition and app submitted to same.
After enabling debug was able to find the below
- From AM the container ask is for OFF-SWITCH
- RM allocating all containers to NODE_LOCAL as shown in logs below.
- So since I was having about 500 containers time taken was about – 6 minutes to allocate 1st map after AM allocation.
- Tested with about 1K maps using PI job took 17 minutes to allocate next container after AM allocation
Once 500 container allocation on NODE_LOCAL is done the next container allocation is done on OFF_SWITCH
(Consumes about 6 minutes)