Major Description
SAMZA-348 has a detailed design proposal on how we can configure Samza via a stream. Part of this work involves converting the SamzaContainer to retrieve its information via an HTTP/JSON request, rather than from JSON encoded environment variables.
The three items that we'll need to serve via a "job coordinator" will be the job's config, the job's container:task:SSP mappings, and the job's task:changelog partition mappings.
We'll also need to assign each container a unique ID, so that it can retrieve its task:SSP mappings (and thus, its task:changelog partition mappings).
This ticket does not encompass creating a "job coordinator". Instead, we'll just start the HTTP server in three places: the ThreadJob, the ProcessJob, and the YARN AM. In all three cases, a URL will be set via an environment variable, which the SamzaContainer will receive, and use to retrieve its information.