[YUNIKORN-317] Cache/Scheduler modules need to be combined to a single module

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10
• Component/s: core - cache, core - scheduler, documentation
• Labels: pull-request-available
• Target Version: 0.10
• Assignee: Wilfred Spiegelenburg
• Reporter: Wangda Tan
• Created: 26/Jul/20 22:36
• Updated: 21/Jan/22 21:48
• Resolved: 01/Dec/20 09:35

Description

Now we have two modules inside core:

• Cache
• Scheduler

Between Cache and Scheduler, they communicate with each other using async events, which causes lots of inconsistent issues. (Such as YUNIKORN-169). 

Also, it complex logic a lot, there're 10 out of 19 events are only between cache and scheduler.

Issue Links

• breaks: YUNIKORN-513 ApplicationState remains in Accepted after recovery
• causes: YUNIKORN-480 root queue max resource gets reset on config load
• causes: YUNIKORN-481 deadlock when removing allocation ask
• contains: YUNIKORN-434 Scheduler crashes when the queueName in pod spec doesn't contain DOT
• fixes: YUNIKORN-230 Incorrect application state returned for "ws/v1/apps" REST call
• fixes: YUNIKORN-77 De-dup event queue methods inside yunikorn-core, and improve logs
• fixes: YUNIKORN-342 Data race in QueueInfo#addChildQueue
• incorporates: YUNIKORN-473 Update doc from cache removal
• is depended upon by: YUNIKORN-402 Make sure when there is no allocation in an app, the app state is "Waiting".
• is duplicated by: YUNIKORN-288 With StateAware Sorting policy enabled, Race condition while moving the apps from accepted to starting state
• supercedes: YUNIKORN-202 race detected in TestBasicSchedulerAutoAllocation
• links to: GitHub Pull Request #33
• links to: GitHub Pull Request #196
• links to: GitHub Pull Request #221