[KYLIN-5595] [kylin 5.0] Launch Job Node not initialize spark session issue

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.0-beta
• Fix Version/s: 5.0-beta
• Component/s: Job Engine
• Assignee: Li Can
• Reporter: Li Can
• Created: 28/Jun/23 07:34
• Updated: 23/Aug/23 07:51
• Resolved: 23/Aug/23 07:51

Description

   Saving model will execute  'checkFlatTableSql'  method at job node, and it will not skip the step by default. When execute 'checkFlatTableSql' method, it will initialize spark session if the job node just started, the process of getting spark session costs too much time.

  The pic 87 shows that get spark session costs more than 63s, and the execution of checking sql costs more than 2s, it is not friendly for saving model first time after node launched, and it is also unreasonable.

  So I suggest that the job node's process of  spark session initialization should be consistent with the query node, it means that the spark session should be initialized as the node just started. And the spark session is a singleton model, just need once initialization, as the pic 88 display.