[SPARK-17288] Spark sbin script on windows support

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: Deploy, Windows
• Labels: bulk-closed windows
• Assignee: Unassigned
• Reporter: wei wu
• Created: 29/Aug/16 09:51
• Updated: 21/May/19 04:33
• Resolved: 21/May/19 04:33

Description

We develop a prototype for spark sbin scripts for windows system. And we have done some verify work based on spark 1.6.1 and windows 7. And we want to contribute. We will submit a PR later. The prototype code mainly include:
1. spark-env.cmd Set spark environments on windows
2. spark-config.cmd Set spark configured environment, just like SPARK_CONF_DIR
3. start-master.cmd Start master daemon
4. start-slave.cmd Start slave daemon
5. start-thriftserver.cmd Start the thrift server daemon
6. start-history-server.cmd Start the history server daemon
7. start-shuffle-service.cmd Start the shuffle service daemon
8. spark-daemon.cmd The common script for windows daemon start
9. stop-master.cmd Stop the master daemon
10. stop-slave.sh Stop the slave daemon
11. stop-history-server.cmd Stop the history server daemon
12. stop-shuffle-service.cmd Stop the shuffle service daemon
13. stop-thriftserver.cmd Stop the thrift server daemon

The daemon start logs will be stored in %SPARK_HOME%\logs, and we support he scrolling logs, default is 5.
The windows daemon pid is stored in %SPARK_HOME%\pid, It is used to stop the windows daemon.

The sbin windows scripts code is helpful to expand spark using on windows.