you cant easily let yarn spin up arbitrary linux containers willy-nilly anymore wihtout kerberos
Another solution - perhaps radical - would be to drop the support of YARN completely. After all, Spark works fine as a standalone system, hence a superior MapReduce model will be supported. When we add Gridgain, which doesn't care about YARN neither, we'll have in-memory OLTP and other things. Perhaps, this will help to finally produce a sane implementation of LinuxTaskController?