[SPARK-25822] Fix a race condition when releasing a Python worker

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.2
• Fix Version/s: 2.3.3, 2.4.0
• Component/s: PySpark
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 24/Oct/18 17:22
• Updated: 26/Oct/18 04:56
• Resolved: 26/Oct/18 04:56

Description

There is a race condition when releasing a Python worker. If "ReaderIterator.handleEndOfDataSection" is not running in the task thread, when a task is early terminated (such as "take(N)"), the task completion listener may close the worker but "handleEndOfDataSection" can still put the worker into the worker pool to reuse.

https://github.com/zsxwing/spark/commit/0e07b483d2e7c68f3b5c3c118d0bf58c501041b7 is a patch to reproduce this issue.

I also found a user reported this in the mail list: http://mail-archives.apache.org/mod_mbox/spark-user/201610.mbox/%3CCAAUq=[e-mail]%3E

Issue Links

• links to: [Github] Pull Request #22816 (zsxwing)
• links to: [Github] Pull Request #22816 (zsxwing)