[FLINK-18157] Jobstore size check compares against offHeapMemory

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Nicholas Jiang
• Reporter: Chesnay Schepler
• Created: 05/Jun/20 11:47
• Updated: 09/Jun/20 16:42
• Resolved: 09/Jun/20 13:51

Description

Setting jobmanager.memory.off-heap.size to 0 results in this confusing error:

[] - Loading configuration property: jobmanager.memory.process.size, 2000m
[] - Loading configuration property: jobmanager.memory.heap.size, 1500m
[] - Loading configuration property: jobmanager.memory.jvm-overhead.min, 100m
[] - Loading configuration property: jobmanager.memory.jvm-overhead.max, 350m
[] - Loading configuration property: jobmanager.memory.off-heap.size, 0m
[] - The configured or derived JVM heap memory size (jobstore.cache-size: 0 bytes) is less than the configured or default size of the job store cache (jobmanager.memory.heap.size: 50.000mb (52428800 bytes))

According to the documentation the jobstore uses the heap though.

private static JobManagerFlinkMemory createJobManagerFlinkMemory(
		Configuration config,
		MemorySize jvmHeap,
		MemorySize offHeapMemory) {
	verifyJvmHeapSize(jvmHeap);
	verifyJobStoreCacheSize(config, offHeapMemory);
	return new JobManagerFlinkMemory(jvmHeap, offHeapMemory);
}

private static void verifyJvmHeapSize(MemorySize jvmHeapSize) {
	if (jvmHeapSize.compareTo(JobManagerOptions.MIN_JVM_HEAP_SIZE) < 1) {
		LOG.warn(
			"The configured or derived JVM heap memory size ({}) is less than its recommended minimum value ({})",
			jvmHeapSize.toHumanReadableString(),
			JobManagerOptions.MIN_JVM_HEAP_SIZE);
	}
}

Issue Links

• Blocked: FLINK-18214 Incorrect warning if jobstore.cache-size exceeds heap size
• is a child of: FLINK-17973 Test memory configuration of Flink cluster
• links to: GitHub Pull Request #12516