[SPARK-40739] "sbt packageBin" fails in cygwin or other windows bash session

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.4.0
• Component/s: Build, Windows
• Labels: bash cygwin mingw msys2, windows

Environment

The problem occurs in Windows if sbt is started from a (non-WSL) bash session.

See the spark PR link for detailed symptoms.

• Flags: Important
• External issue URL: https://github.com/apache/spark/pull/38228
• External issue ID: 38228

Language

bash java

• Assignee: Phil Walker
• Reporter: Phil Walker
• Created: 10/Oct/22 20:34
• Updated: 24/Oct/22 13:29
• Resolved: 24/Oct/22 13:29

Description

 In a Windows SHELL environment, such as cygwin or msys2/mingw64, etc,  Core.settings in project/SparkBuild.scala calls the wrong bash.exe if WSL bash is present (typically at C:\Windows), causing a build failure.  This occurs even though the proper bash.exe is in the PATH ahead of WSL bash.exe.

This is fixed by spark PR 38228

There are 3 parts to the fix, implemented in project/SparkBuild.scala :

• determine the absolute path of the first bash.exe in the command line. 
• determine the build environment (e.g., Linux, Darwin, CYGWIN, MSYS2, etc.)
• For Windows SHELL environments, the first argument to the spawned Process is changed from "bash" to the absolute path.

Issue Links

• links to: [Github] Pull Request #38167 (philwalk)
• links to: [Github] Pull Request #38167 (philwalk)
• links to: [Github] Pull Request #38228 (philwalk)