Details
Blocker Description
This is related to the recent changes in AURORA-920. When scheduler starts it goes through all tasks in the log and generates a multitude of events (e.g. TaskStateChange.initialized(task)). For the new AsyncEventBus we use a Executors.newCachedThreadPool(), which does not have an upper bound on how many threads it can create when needed (well, technically it's bounded by Integer.MAX_VALUE). That leads to an out of control thread creation and eventual out of memory JVM crash.
Use ThreadPoolExecutor with a bounded maximumPoolSize to control the memory consumption. The AsyncEventBus uses a simple ConcurrentLinkedQueue to enqueue events, so there is no risk in timing out due to slower event processing.