[SPARK-21877] Windows command script can not handle quotes in parameter

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: Deploy, Windows

Environment

Spark version: spark-2.2.0-bin-hadoop2.7
Windows version: Windows 10

• Assignee: Xiaokai Zhao
• Reporter: Xiaokai Zhao
• Created: 31/Aug/17 00:44
• Updated: 12/Dec/22 18:10
• Resolved: 06/Oct/17 14:39

Description

All the windows command scripts can not handle quotes in parameter.

Run a windows command shell with parameter which has quotes can reproduce the bug:

C:\Users\meng\software\spark-2.2.0-bin-hadoop2.7> bin\spark-shell --driver-java-options " -Dfile.encoding=utf-8 "
'C:\Users\meng\software\spark-2.2.0-bin-hadoop2.7\bin\spark-shell2.cmd" --driver-java-options "' is not recognized as an internal or external command,
operable program or batch file.

Windows recognize "--driver-java-options" as part of the command.

All the Windows command script has the following code have the bug.

cmd /V /E /C "<other command>" %*

Issue Links

• links to: [Github] Pull Request #19090 (minixalpha)