Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
For rolling out jobs to an external cluster, we currently have 3 choices:
a) Manual submission with Web Interface
b) Automatic/Manual submission with CLClient
c) Automatic submission with custom client
I propose to add a way to submit jobs automatically through a HTTP Rest Interface. Among other benefits, this extension allows an automatic submission of jobs through a restrictive proxy.
Rough idea:
The web interface would offer a REST entry point for example /jobs. POSTing to this entry point allows the submission of a new job and returns the job URL. http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html
GETting the job URL returns a small status.
DELETING the job URL aborts the job.
GETting on the /jobs returns a list of active and scheduled jobs.
Since Flink already has a Jetty web server and uses Json for other services, the basic extension should require low effort. It would help Flink to be used inside larger corporations and align the interfaces with the other state-of-the-art MapReduce systems (s3, HDFS, HBase all have HTTP interface).