[FLINK-20973] failed to submit flink-on-yarn job on Windows due to error of classPath

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.10.1, 1.11.3
• Component/s: Client / Job Submission, Deployment / YARN
• Assignee: Unassigned
• Reporter: kunnni
• Created: 14/Jan/21 10:35
• Updated: 14/Jan/21 11:30
• Resolved: 14/Jan/21 11:29

Description

description: When I submitted flink-on-yarn job on Windows (Operating system of yarn cluster is Linux), I got error message: 

Error: Could not find or load main class org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint

cause: Windows has different classPath delimiter with Linux as in Windows is " ; ", but in Linux is " : ".  the main cause is that classPath was built on Windows, but was splitted on Linux. 

solution : replace value of  File.pathSeparator with ApplicationConstants.CLASS_PATH_SEPARATOR in class of org.apache.flink.yarn.YarnClusterDescriptor and org.apache.flink.yarn.Utils.

 

sourceCode is displayed below：

private ApplicationReport startAppMaster(......) throws Exception {
 ......
 // classpath assembler
 StringBuilder classPathBuilder = new StringBuilder();
 if (userJarInclusion == YarnConfigOptions.UserJarInclusion.FIRST) {
 for (String userClassPath : userClassPaths)
{ classPathBuilder.append(userClassPath).append(File.pathSeparator); }
}
 for (String classPath : systemClassPaths)
{ classPathBuilder.append(classPath).append(File.pathSeparator); }
......
 // Setup CLASSPATH and environment variables for ApplicationMaster
 final Map<String, String> appMasterEnv = new HashMap<>();
 // set user specified app master environment variables
 appMasterEnv.putAll(
 BootstrapTools.getEnvironmentVariables(ResourceManagerOptions.CONTAINERIZED_MASTER_ENV_PREFIX, configuration));
 // set Flink app class path
 appMasterEnv.put(YarnConfigKeys.ENV_FLINK_CLASSPATH, classPathBuilder.toString());
 ......
 }

 

 

 

Issue Links

• is related to: FLINK-16683 Remove scripts for starting Flink on Windows
• is related to: FLINK-17858 Yarn mode, windows and linux environment should be interlinked