[FLINK-31140] Load additional dependencies in operator classpath

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: kubernetes-operator-1.5.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Tamir Sagi
• Created: 20/Feb/23 12:36
• Updated: 02/Mar/23 15:09
• Resolved: 02/Mar/23 15:09

Description

To date is is not possible to add additional jars to operator classpath. 

In our case, We have a Kafka appender with custom layout that works with IAM authentication and SSL along with AWS MSK.

log4j.properties file

 

appender.kafka.type = Kafka
appender.kafka.name = Kafka
appender.kafka.bootstrap.servers = <brokers>
appender.kafka.topic = <topic>
appender.kafka.security.protocol = SASL_SSL
appender.kafka.sasl.mechanism = AWS_MSK_IAM
appender.kafka.sasl.jaas.config = software.amazon.msk.auth.iam.IAMLoginModule required;
appender.kafka.sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler
appender.kafka.layout.type = CustomJsonLayout
appender.kafka.layout.class = our.package.layouts.CustomJsonLayout 
appender.kafka.layout.service_name = <service-name>

if CustomLayout is not present in classpath the logger fails to start.

main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.KafkaAppender for element Kafka: java.lang.NullPointerException java.lang.NullPointerException

furthermore, all the essential AWS dependencies must be added to path.

 

To support additional jars I needed to add them and then replace the following lines

https://github.com/apache/flink-kubernetes-operator/blob/main/docker-entrypoint.sh#L32

https://github.com/apache/flink-kubernetes-operator/blob/main/docker-entrypoint.sh#L37

using 'sed' command.

LOGGER_JAR in that case is an uber jar contains all the necessary jars.

ENV OPERATOR_LIB=/opt/flink/operator-lib
RUN mkdir -p $OPERATOR_LIB
COPY target/$LOGGER_JAR $OPERATOR_LIB
USER root

RUN sed -i 's/java -cp /java -cp $USER_DEPENDENCIES_DIR/*:/' /docker-entrypoint.sh

USER flink

It works great but not ideal and IMO should not be handled that way.

 

The idea of that ticket is to allow users to add additional jars into specific location which is added to classpath while Kubernetes operator starts.

I'd like to add the 'OPERATOR_LIB' ENV and create that folder (/opt/flink/operator-lib) as well in root Dockerfile.

along with a minor modification in 'docker-entrypoint.sh' (Add  $OPERATOR_LIB to 'cp' command)

 

Once it's supported , the only thing users have to do is copy all dependencies to that folder ($OPERATOR_LIB) while building custom image

I tested it locally and it seems to be working.

Issue Links

• relates to: FLINK-30115 Include log4j-layout-template-json jar in the default operator image
• links to: GitHub Pull Request #536