[SPARK-26670] worker.py uses resource python library, which is not available for windows

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.4.0
• Component/s: PySpark
• Assignee: Unassigned
• Reporter: René Buffat
• Created: 20/Jan/19 14:36
• Updated: 12/Dec/22 18:10
• Resolved: 21/Jan/19 01:07

Description

worker.py in spark 2.4.0 uses the resource python library, which is only available for unix.

 

I'm trying out to run sample code for pyspark on Windows. It fails with an import error in worker.py for "resource". Thus, the sample code does not work on windows with spark 2.4.0. However, the same code works with spark 2.3.2 (which lacks the "resource" dependency).

Issue Links

• duplicates: SPARK-26080 Unable to run worker.py on Windows