[SPARK-42370] Spark History Server fails to start on CentOS7 aarch64

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.2.3
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Zhiguo Wu
• Created: 07/Feb/23 08:54
• Updated: 07/Feb/23 11:28

Description

When I run `./sbin/start-history-server.sh`

I'll get the error below

 

Although we already use org.openlabtesting.leveldbjni on aarch64, which can works on aarch64,

we still load org.fusesource.hawtjni.runtime.Library on wrong jar file

When we run `export SPARK_DAEMON_JAVA_OPTS=-verbose:class`, we can see the class is loaded from jline-2.14.6.jar where the correct class file is under leveldbjni-all-1.8.jar

 

Incorrect(now):

[Loaded org.fusesource.hawtjni.runtime.Library from file:/yourdir/spark/jars/jline-2.14.6.jar] 

Correct(expected):

[Loaded org.fusesource.hawtjni.runtime.Library from file:/yourdir/spark/jars/leveldbjni-all-1.8.jar] 

Issue Links

• is related to: BIGTOP-3907 Spark History Server fails to start on CentOS7 aarch64