[KYLIN-4163] CreateFlatHiveTableStep has not yarn app url when hive job running - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: v3.0.0-alpha
Fix Version/s: v3.0.0
Component/s: Job Engine, Web
Labels:
None

Description

CreateFlatHiveTableStep has yarn app url on the monitor web page only when job finished, but SparkExecutable has yarn app url when job running.
this is because of SparkExecutable`s logger has logger listener:

final PatternedLogger patternedLogger = new PatternedLogger(logger, new PatternedLogger.ILogListener() {
 @Override
 public void onLogEvent(String infoKey, Map<String, String> info) {
 // only care three properties here
 if (ExecutableConstants.SPARK_JOB_ID.equals(infoKey)
 || ExecutableConstants.YARN_APP_ID.equals(infoKey)
 || ExecutableConstants.YARN_APP_URL.equals(infoKey)) {
 getManager().addJobInfo(getId(), info);
 }
 }
 });

sometimes creating flat hive table hangs, so user wants to have yarn app url when hive job running like attachment.