Major Description
The config.sh script uses the following shell-script function to build the FLINK_CLASSPATH variable from a listing of JAR files in the $FLINK_LIB_DIR directory:
constructFlinkClassPath() {
while read -d '' -r jarfile ; do
if [[ $FLINK_CLASSPATH = "" ]]; then
FLINK_CLASSPATH="$jarfile";
else
FLINK_CLASSPATH="$FLINK_CLASSPATH":"$jarfile"
fi
done < <(find "$FLINK_LIB_DIR" ! -type d -name '*.jar' -print0)
echo $FLINK_CLASSPATH
}
The find command as specified will return files in directory-order, which varies by OS and filesystem.
The inconsistent ordering of directory contents caused problems for me when installing a Flink Docker image onto new machine with a newer version of Docker and different filesystem (UFS). The differences in the Docker filesystem implementation led to different ordering of the directory contents; this affected the FLINK_CLASSPATH ordering and generated very puzzling NoClassNotFoundException errors when running my Flink application.
This should be addressed by deterministically ordering JAR files added to the FLINK_CLASSPATH.
Attachments
Issue Links
- is duplicated by
-
FLINK-6403 constructFlinkClassPath produces nondeterministic classpath
-
- Closed
-
- links to
