[FLINK-4326] Flink start-up scripts should optionally start services on the foreground

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.3
• Fix Version/s: 1.2.1, 1.3.0
• Component/s: Deployment / Scripts
• Assignee: Greg Hogan
• Reporter: Elias Levy
• Created: 06/Aug/16 00:01
• Updated: 02/Oct/19 17:45
• Resolved: 08/Mar/17 10:16

Description

This has previously been mentioned in the mailing list, but has not been addressed. Flink start-up scripts start the job and task managers in the background. This makes it difficult to integrate Flink with most processes supervisory tools and init systems, including Docker. One can get around this via hacking the scripts or manually starting the right classes via Java, but it is a brittle solution.

In addition to starting the daemons in the foreground, the start up scripts should use exec instead of running the commends, so as to avoid forks. Many supervisory tools assume the PID of the process to be monitored is that of the process it first executes, and fork chains make it difficult for the supervisor to figure out what process to monitor. Specifically, jobmanager.sh and taskmanager.sh should exec flink-daemon.sh, and flink-daemon.sh should exec java.

Issue Links

• is duplicated by: FLINK-5634 Flink should not always redirect stdout to a file.
• relates to: FLINK-4208 Support Running Flink processes in foreground mode
• links to: GitHub Pull Request #3351
• links to: GitHub Pull Request #3492