[FLINK-32909] The jobmanager.sh pass arguments failed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.2, 1.18.0, 1.17.1
• Fix Version/s: 1.18.0, 1.16.3, 1.17.2
• Component/s: Deployment / Scripts
• Labels: pull-request-available
• Assignee: Alex Wu
• Reporter: Alex Wu
• Created: 22/Aug/23 09:28
• Updated: 01/Sep/23 09:20
• Resolved: 01/Sep/23 09:20

Description

I' m trying to use the jobmanager.sh script to create a jobmanager instance manually, and I need to pass arugments to the script dynamically, rather than through flink-conf.yaml. But I found that I didn't succeed in doing that when I commented out all configurations in the flink-conf.yaml,  I typed command like:

 

./bin/jobmanager.sh start -D jobmanager.memory.flink.size=1024m -D jobmanager.rpc.address=xx.xx.xx.xx -D jobmanager.rpc.port=xxx -D jobmanager.bind-host=0.0.0.0 -D rest.address=xx.xx.xx.xx -D rest.port=xxx -D rest.bind-address=0.0.0.0

but I got some errors below:

 

[ERROR] The execution result is empty.
[ERROR] Could not get JVM parameters and dynamic configurations properly.
[ERROR] Raw output from BashJavaUtils:
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
Exception in thread "main" org.apache.flink.configuration.IllegalConfigurationException: JobManager memory configuration failed: Either required fine-grained memory (jobmanager.memory.heap.size), or Total Flink Memory size (Key: 'jobmanager.memory.flink.size' , default: null (fallback keys: [])), or Total Process Memory size (Key: 'jobmanager.memory.process.size' , default: null (fallback keys: [])) need to be configured explicitly.
        at org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobManagerProcessUtils.java:78)
        at org.apache.flink.runtime.util.bash.BashJavaUtils.getJmResourceParams(BashJavaUtils.java:98)
        at org.apache.flink.runtime.util.bash.BashJavaUtils.runCommand(BashJavaUtils.java:69)
        at org.apache.flink.runtime.util.bash.BashJavaUtils.main(BashJavaUtils.java:56)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Either required fine-grained memory (jobmanager.memory.heap.size), or Total Flink Memory size (Key: 'jobmanager.memory.flink.size' , default: null (fallback keys: [])), or Total Process Memory size (Key: 'jobmanager.memory.process.size' , default: null (fallback keys: [])) need to be configured explicitly.
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.failBecauseRequiredOptionsNotConfigured(ProcessMemoryUtils.java:129)
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:86)
        at org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfig(JobManagerProcessUtils.java:83)
        at org.apache.flink.runtime.jobmanager.JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(JobM 

It seems to remind me to configure memory for jobmanager instance explicitly, but I had already passed the jobmanager.memory.flink.size parameter. So I debug the script, and found a spelling error in the jobmanager.sh script at line 54:

 

parseJmArgsAndExportLogs "${ARGS[@]}"

the uppercase "${ARGS[@]}" is a wrong variable name here from a contextual perspective, and causing an empty string passed to the function. I changed to "${args[@]}" and It works fine.

 

 

 

Issue Links

• links to: GitHub Pull Request #23271