In a YARN cluster you can't predict where services will come up -or on what ports. The services need to work those things out as they come up and then publish them somewhere.
Applications need to be able to find the service instance they are to bond to -and not any others in the cluster.
Some kind of service registry -in the RM, in ZK, could do this. If the RM held the write access to the ZK nodes, it would be more secure than having apps register with ZK themselves.
|1.||RM to support YARN registry||Patch Available|
|2.||distributed shell & tests to use registry||Open|
|3.||MiniYARNCluster to (optionally) add MicroZookeeper service||Open|
|4.||Provide Read Write REST view of YARN registry and client compatible with existing Registry API||Open||Unassigned|
|5.||TTL for YARN Registry SRV records||Open||Unassigned|
|6.||Collision-free unique bindings & refresh APIs for service records||Open||Unassigned|
|7.||TTL & identity aware read cache for the SRV records||Open||Unassigned|