Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
1.9.0
-
None
-
None
Description
Currently, we create JobGraph by the following process
- if the cluster start in job mode, we create JobGraph by PackagedProgramUtils#createJobGraph and deploy a job cluster
- if the cluster start in session mode, we create JobGraph and submit it by CliFrontend#executeProgram, which internally the same as above but using ContextEnvironment instead of OptimizerPlanEnvironment.
ContextEnvironment not only create the job graph but also submit it. However, the processes of JobGraph creation in job mode and session mode are similar. That means, we can unify the process by always create JobGraph by PackagedProgramUtils#createJobGraph. And,
- in job mode, deploy job cluster with the JobGraph
- in session mode, submit the JobGraph to the session cluster
From a higher view, it is helpful for a common view of job submission in both job and session mode and give opportunities to refactor legacy client codes.