[ZEPPELIN-5516] Regression Testing:when yarn-application model, some functions cannot be work normally

• Type: Test
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.1
• Component/s: interpreter-setting, zeppelin-interpreter, zeppelin-server

Environment

• Run zeppelin with yarn-application mode

• Assignee: Unassigned
• Reporter: vmaster.cc
• Created: 13/Sep/21 06:32
• Updated: 10/Feb/23 01:58
• Resolved: 10/Feb/23 01:58

Description

after change to yarn-application mode, some functions cannot work normally, even effect other mudules, eg:

• Q1、when use default config of 'zeppelin.flink.uiWebUrl', it will show a strange Flink Web　URL in zeppelin paragraph running UI. eg: `zeppelin.domain`/false/#/job/jobId. or else localhost display.

 

private def getDisplayedJMWebUrl(yarnAppId: String): String = {
  // `zeppelin.flink.uiWebUrl` is flink jm url template, {{applicationId}} will be replaced
  // with real yarn app id.
  val flinkUIWebUrl = properties.getProperty("zeppelin.flink.uiWebUrl")
  if (StringUtils.isNotBlank(flinkUIWebUrl)) {
    flinkUIWebUrl.replace("{{applicationId}}", yarnAppId)
  } else {
    this.jmWebUrl
  }
}

// 2
flink/flink-scala-parent/src/main/resources/interpreter-setting.json
"zeppelin.flink.uiWebUrl": {
  "envName": null,
  "propertyName": null,
  "defaultValue": false,
  "description": "User specified Flink JobManager url, it could be used in remote mode where Flink cluster is already started, or could be used as url template, e.g. https://knox-server:8443/gateway/cluster-topo/yarn/proxy/{{applicationId}}/ where {{applicationId}} would be replaced with yarn app id",
  "type": "string"
},

 

 

• Q2、when recovery is enabled, some functions different even cannot work after zeppelin restarted.
  • S1、it will report NPE or RuntimeException, Cannot recover unless restarted (But some times it's work ok)
  • S2、cannot rerun conf, env it not changed, it will report :java.io.IOException: Can not change interpreter properties when interpreter process has already been launched
  • S3、Flink Web UI URL button lost,see follow screenshot:

 

• Q3、yarn model `hadoop classpath` need only, but yarn-application mode we must config HADOOP_CLASSPATH env

 

If there are problems that have not be clearly described, you can contact me at any time:

email: [e-mail]