Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Job coordinator (AM) High availability (HA) will bring up a new AM which can connect to already running containers of the job. This ticket is to capture the physical execution environment container id (ex:yarn container id "container_123") mapping to samza logical processor id (ex: 0,1,..).
Each container should write its mapping of physical id to logical id and AM should read this mapping. This mapping will be written to coordinator stream.
New AM coming up under HA will use this mapping to associate the already running yarn containers list handed to it from the ResourceManager. New AM using this mapping is part of future work and not a deliverable for this task.