[HADOOP-16816] Please remove or minimize need for native libraries "winutils" to run Spark locally on Windows.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.1, 3.1.3
• Fix Version/s: HADOOP-13223
• Component/s: auth, bin, common, native

Environment

Windows

• Assignee: Unassigned
• Reporter: mark pettovello
• Created: 21/Jan/20 01:23
• Updated: 21/Jan/20 14:33
• Resolved: 21/Jan/20 14:33

Description

Please remove the need for building and using custom native libraries "winutils" to run Spark on Windows systems just to run Spark & similar.

Reference: https://github.com/steveloughran/winutils

If native libraries are needed for performance benefits then please try to load the native libraries and then if not found then provide and use a functional equivalent Java implementation such that Spark will build and run applications using only Java. 

Issue Links

• duplicates: HADOOP-13223 winutils.exe is a bug nexus and should be killed with an axe.
• links to: GitHub Pull Request #1812