[SPARK-28582] Pyspark daemon exit failed when receive SIGTERM on py3.7

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.3
• Fix Version/s: 2.3.4, 2.4.4, 3.0.0
• Component/s: PySpark
• Assignee: Weichen Xu
• Reporter: Weichen Xu
• Created: 31/Jul/19 15:45
• Updated: 12/Dec/22 18:10
• Resolved: 03/Aug/19 01:36

Description

Pyspark daemon exit failed when receive SIGTERM on py3.7.

We can run test on py3.7 like

python/run-tests --python-executables=python3.7 --testname "pyspark.tests.test_daemon DaemonTests"

Will fail on test "test_termination_sigterm". And we can see daemon process do not exit.

This issue happen on py3.7 but lower version python works fine.

Issue Links

• links to: GitHub Pull Request #25315
• links to: GitHub Pull Request #25343