[FLINK-26537] Allow disabling StatefulFunctionsConfigValidator validation for classloader.parent-first-patterns.additional

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: Stateful Functions
• Labels: pull-request-available
• Assignee: Fil Karnicki
• Reporter: Fil Karnicki
• Created: 08/Mar/22 16:48
• Updated: 16/Mar/22 12:07
• Resolved: 16/Mar/22 12:07

Description

For some deployments of stateful functions onto existing, shared clusters, it is impossible to tailor which classes exist on the classpath. An example would be a Cloudera Flink cluster, which adds protobuf-java classes that clash with statefun ones.

Stateful functions require the following flink-config.yaml setting:

classloader.parent-first-patterns.additional: org.apache.flink.statefun;org.apache.kafka;com.google.protobuf 

In the case of the cloudera flink cluster, this will cause old, 2.5.0 protobuf classes to be loaded by statefun, which causes MethodNotFound exceptions. 

The idea is to allow disabling of the validation below, if some config setting is present in the global flink configuration, for example: statefun.validation.parent-first-classloader.disable=true

 

private static void validateParentFirstClassloaderPatterns(Configuration configuration) {
  final Set<String> parentFirstClassloaderPatterns =
      parentFirstClassloaderPatterns(configuration);
  if (!parentFirstClassloaderPatterns.containsAll(PARENT_FIRST_CLASSLOADER_PATTERNS)) {
    throw new StatefulFunctionsInvalidConfigException(
        CoreOptions.ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL,
        "Must contain all of " + String.join(", ", PARENT_FIRST_CLASSLOADER_PATTERNS));
  }
} 

 

Then, we wouldn't need to contain com.google.protobuf in classloader.parent-first-patterns.additional: and it would be up to the statefun user to use protobuf classes which are compatible with their version of statefun.

 

 

Issue Links

• links to: GitHub Pull Request #307