Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4163

CreateFlatHiveTableStep has not yarn app url when hive job running

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: v3.0.0-alpha
    • Fix Version/s: v3.0.0
    • Component/s: Job Engine, Web
    • Labels:
      None

      Description

      CreateFlatHiveTableStep has yarn app url on the monitor web page only when job finished, but SparkExecutable has yarn app url when job running.
      this is because of SparkExecutable`s logger has logger listener:

      final PatternedLogger patternedLogger = new PatternedLogger(logger, new PatternedLogger.ILogListener() {
       @Override
       public void onLogEvent(String infoKey, Map<String, String> info) {
       // only care three properties here
       if (ExecutableConstants.SPARK_JOB_ID.equals(infoKey)
       || ExecutableConstants.YARN_APP_ID.equals(infoKey)
       || ExecutableConstants.YARN_APP_URL.equals(infoKey)) {
       getManager().addJobInfo(getId(), info);
       }
       }
       });

      sometimes creating flat hive table hangs, so user wants to have yarn app url when hive job running  like attachment.

        Attachments

        1. KYLIN-4163.master.001.patch
          4 kB
          chuxiao
        2. flathivetablerunning图.jpg
          48 kB
          chuxiao

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                xiaoge chuxiao
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: