[BIGTOP-3814] Deploying Spark ThriftServer fails because it's launched too early

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0
• Component/s: deployment, spark
• Assignee: Kengo Seki
• Reporter: Kengo Seki
• Created: 13/Sep/22 22:13
• Updated: 15/Sep/22 07:33
• Resolved: 15/Sep/22 07:33

Description

I tried to deploy Spark on Rocky 8 using Docker provisioner and came across the following failure.

[vagrant@rocky8 docker]$ ./gradlew allclean spark-pkg repo -Dbuildwithdeps=true
[vagrant@rocky8 docker]$ cd provisioner/docker
[vagrant@rocky8 docker]$ ./docker-hadoop.sh -d -C config_rockylinux-8.yaml -r file:///bigtop-home/output -G -k hdfs,yarn,spark -c 3

...

Notice: /Stage[main]/Spark::Spark_thriftserver/Package[spark-thriftserver]/ensure: created
Error: Systemd start for spark-thriftserver failed!
journalctl log for spark-thriftserver:
-- Logs begin at Tue 2022-09-13 10:48:23 UTC, end at Tue 2022-09-13 10:55:42 UTC. --
Sep 13 10:55:35 c93a93f857f8 systemd[1]: Starting LSB: Spark thriftserver...
Sep 13 10:55:35 c93a93f857f8 spark-thriftserver[36957]: Starting Spark thriftserver (spark-thriftserver):[  OK  ]
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session opened for user spark by (uid=0)
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session closed for user spark
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Control process exited, code=exited status=1
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Failed with result 'exit-code'.
Sep 13 10:55:42 c93a93f857f8 systemd[1]: Failed to start LSB: Spark thriftserver.

Error: /Stage[main]/Spark::Spark_thriftserver/Service[spark-thriftserver]/ensure: change from 'stopped' to 'running' failed: Systemd start for spark-thriftserver failed!
journalctl log for spark-thriftserver:
-- Logs begin at Tue 2022-09-13 10:48:23 UTC, end at Tue 2022-09-13 10:55:42 UTC. --
Sep 13 10:55:35 c93a93f857f8 systemd[1]: Starting LSB: Spark thriftserver...
Sep 13 10:55:35 c93a93f857f8 spark-thriftserver[36957]: Starting Spark thriftserver (spark-thriftserver):[  OK  ]
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session opened for user spark by (uid=0)
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session closed for user spark
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Control process exited, code=exited status=1
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Failed with result 'exit-code'.
Sep 13 10:55:42 c93a93f857f8 systemd[1]: Failed to start LSB: Spark thriftserver.

Notice: /Stage[main]/Spark::Spark_thriftserver/Service[spark-thriftserver]: Triggered 'refresh' from 3 events

STS uses HDFS so it should be launched after initializing HDFS.

Issue Links

• links to: https://github.com/apache/bigtop/pull/1008