Description
The config.sh script uses the following shell-script function to build the FLINK_CLASSPATH variable from a listing of JAR files in the $FLINK_LIB_DIR directory:
constructFlinkClassPath() { while read -d '' -r jarfile ; do if [[ $FLINK_CLASSPATH = "" ]]; then FLINK_CLASSPATH="$jarfile"; else FLINK_CLASSPATH="$FLINK_CLASSPATH":"$jarfile" fi done < <(find "$FLINK_LIB_DIR" ! -type d -name '*.jar' -print0) echo $FLINK_CLASSPATH }
The find command as specified will return files in directory-order, which varies by OS and filesystem.
The inconsistent ordering of directory contents caused problems for me when installing a Flink Docker image onto new machine with a newer version of Docker and different filesystem (UFS). The differences in the Docker filesystem implementation led to different ordering of the directory contents; this affected the FLINK_CLASSPATH ordering and generated very puzzling NoClassNotFoundException errors when running my Flink application.
This should be addressed by deterministically ordering JAR files added to the FLINK_CLASSPATH.
Attachments
Issue Links
- is duplicated by
-
FLINK-6403 constructFlinkClassPath produces nondeterministic classpath
- Closed
- links to